Swords might have not been expensive, but peasants didn't have a lot of pocket change. That is the big reason why the average guy didn't carry swords - it would be a ridiculous expense, and ridiculous expenses weren't something peasants had the ability to go and do most of the time. Better to use an unnatural windfall to buy your wife a dress, or save it for a daughter's dowry, and just defend yourself with whatever other sharp object you inevitably own as a working man.