Some worlds have people missing already. Some might have them disappear when our characters get there. Others might remain completely unaffected the entire time. 

And don't worry about the characters. I was only going to put up the major ones. So servants wouldn't really count. Unless there's something special about them. If that makes any sense.