The judges wear clothing beneath the robes but nothing more than cloth tunics, breeches, coats, etc - nothing providing significant defense.  Assume the robes are tailored more or less to the individual - it may be loose for movement but not so loose that it becomes cumbersome or restrictive.

If you guys stealthily approach the door you can reach it and perhaps observe the Arbiters without them seeing you, but by entering the room you will immediately alert them and they will be prepared to fight, either because of the ships bumping into each other, or some passing sailors alerted them to the attack.