People eat crops seasonally too. Soldiers, peasents, townspeople, doesn't matter, they all eat crops. Townspeople and raised levies don't produce any crops, and thus don't count toward the population.

@ Nex: Aye, values are quite different this time around. No vassals to take your land, though, which I'm sure is likeable.