I'd imagine that unless there's a courtyard within the walls of the castle itself, the general principle with fortified castles is that there's one(or two, but I think this only has one) outer wall that circles the building and one main building, with perhaps stations and small huts built against the walls, but rarely any true separation to make for individual courtyards. I could be wrong though.

That being said. It's a pretty big space to move around in if it belongs to a king and can fit a full stable and other animal pens in it. Soooo, it's probably entirely possible that they're in the same space and not able to see each other.