Possibly. I was also considering him being like Lyn's mother, and fleeing the throne, but this time the throne would be whatever the Despot sits on. (They have despots in FE, right?)
Maybe I could combine the ideas? He creates the team after the bandit attack, using his influence as a noble to help the others? The only issue I have is deciding whether he owns up to it, or hides his heritage.