After reading what you have to say I understand how strongly you seem to feel about these things.
I don't understand what is wrong with the sentence "Looking as if abandoned" if you could elaborate the problem I will be able to tackle it.

I have changed the guards position of sword to their left, this is so that you understand it was something I simply decided on whilst writing the piece.
Despite this many factors that could be the reason why they would have a sword on there right.
For instance you say that in our history most people where right handed and therefore more right handed sword wielders, but this isn't within our history. For all you know in this universe the majority of people could be left handed, and most guards and soldiers would then be left handed. 
Maybe, as I only described the few guards that where leaving/entering the guard tower, the guards are coming off work, and so place their swords on their right to signify they are off duty. 
Maybe the town guardsman use their left hands for swords as they all have powerful magic which they cast with their dominant right hands. Using the sword as a secondary weapon.

But as I have said above, for simplicity I have edited the original post.