You essentially have to be able to get there in order to exist there (its thanks to the denizens' unique existence that they can exist there)

Its not that you can't go there, its that humans (or anyone of what the denizens call the 'mortal' realm) can't exist there.