I thought you could go to each floor no matter what the level. Cause in the anime, Kirito went to lower floors at a high level.