We could always start early. If anyone wants to join, I guess they can whenever? 
Your decision of course. 4 isn't a bad number of players.