Don't worry, every character is independant of each other, meaning that you can join whenever you want without disrupting the story