He has a tendency to do that XD 
He was more upset that she thought being a servant was like being a chamber maid