Inviting them over may be a good thing, because at least then, they're all in one place, it'll be easier for character development while you both are away, it's a safe place for them and anyone could leave at any time ... just my thoughts