The way I figured it, if your character is human-like and has never been on Earth, they just happened to develop the English language by default. All the Earth-goers are immune to the handwavium unless they decide not to be.