Well, I think that makes it more interesting.

Keep in mind that even if we're far apart, it has not been defined how large spread the different species are, so to speak. So we could /technically/ be far off, and still end up getting attacked by some vestigial members of another species.