The multiverse is basically the vague concept that every place described in Arena character's backstories and all the events that transpire in Arena all happen in different but connected universes. One such connection is the Nexus of Worlds, another is the Conjuncture (which is similar to LeeRoy's reiteration of the Multiverse.) A lot of fighting just takes place on Earth, because plenty of characters share that Universe, but some characters need some sciency-magicky-storyteleportation to get into fights, and the Nexus or the Conjuncture provides it.

That being said, only Arena universes are connected by these things, but if someone were to enter the site with pre-established story they can have their characters 'enter' the RPG connected universes (Multiverse) as if they were entering a whole new dimension, which in a sense they are, as absolutely everything is new to them. They don't even particularly need to utilize the pre-established multiverse tools to do it, they can just get caught up in an accident, thrown into a black hole, w/e. If all the universes of the Arena are connected, it's not that much of a stretch to imagine that the multiverses of all sites are connected, just so long as the people behind the story are.