Also gotta think of it from the perspective of the dominant race in the kingdom, which are humans. Humans, which are known to shit on even other humans for looking ugly n shit. Even if there were orcs/goblins/trolls who weren't violent barbarians, they're still gonna end up as ugly freaks from the human perspective, and thus, they're unlikely to get anywhere in a human dominated society.