He's probably just shy about meeting Gobskag the Great, the famous shaman of the Scarey Face tribe.

Head back in and leave it ambiguous as to what happened imo, that way he can fill in the gaps if he returns, and we're not stuck waiting forever if he doesn't.