Just get the guys with swords to stand near the entrance and chop anyone who isnt allowed to come in