I'm thinking more that there was some catastrophic event in their home world and they were forced to flee it and a handful of them came through the same rift and found this universe at the beginning of it's life and decided to settle here. Others escaped through other rifts to other places, but not many survived the event making our gods very rare, especially in such a large(Relatively speaking) group. The ones who would come through the rift are ones who went to other universes and want to, maybe, in a way, be with their own kind? Most don't stay, feeling out of place in this new society the gods have built, but it's not like out group rejects others, just kinda distant until they are forced to interact. Think of it like they were one of the first universes to develop, and the first to bring about their own downfall, which is why they need a new place to live, but 99% of all universes out there are still in their infancy like the one our group finds themselves in. . . Make more sense?