Oh, I'm sorry if I didn't make this clear, because it's important: if a town is destroyed, that counts as having all its trade routes raided. Meaning, nobody's towns get a wealth upgrade this annum. And you're losing a great deal of tariffs.

Someone destroyed those towns, and it weren't pirates. 

Just thought I'd inform you all.