If it'll take twelve hours to reach Phandalin, we definitely don't have time to go to there first. We need to chase after the goblin that got away. We don't all have to go, if Yorrel would rather stay with the caravan and keep an eye on Maria until we get back, but someone has to go and soon.