The 20/100 analogy just seems to basically be pointing out that if 20 years is 20% of a Gems life, 20% of a Drakkens life is 100 years. It doesn't say anything about development; And surely the description of the Drakkens homeland as being a harsh, brutal place doesn't support the idea of the species having a long development time, they wouldn't have that luxury. If anything they would need to mature quicker.  

As for selection, the Gems aren't actively selecting for those types of genes... The Drakken actually are. Drakken women are rare, they probably marry exclusively the high class Drakken. Gem brides are also exclusively the higher class Drakken. So how come there is a body of common Drakken at all? Surely it should be dwindling?
Well, a very sexually deprived brute just took part in a raid on a town; He decides he doesn't quite like being sexually deprived and sees a few juicy foreign females; He decides he wants to take them home to be his concubines. His commander turns a blind eye; After all, he gets to bugger beautiful Gem girls and Drakken ladies, what does it matter what his men do with foreign dogs?
Now that brute is taking these women, and another one decides he quite likes the look of them. So he basically engages in posturing and, if that fails, a fight; The strongest, fastest, most intimidating and quite possibly the most cunning would be the most likely to make the other back down and keep the women for himself.
In other words, traits of an average, successful common Drakken; Strong, Fast, Tall (intimidation) large horns (intimidation again) Sly.
Now, the biggest and strongest has won; He takes these women home and rapes them through the peace time in order to pass it by; this inadvertently results in a lot of children sharing his genes.

The advantage to defending isn't that nobody knows where your stuff is, its that you force the enemy to come to you; Its much easier to hold a hill or mountain pass than it is to take it, for example.

You actually miss an advantage the Gems have, believe it or not; They're a breadbasket by comparison. They would almost certainly have more people. Unless the Gems are absolute idiots, they at least have some laws regarding levies. Raise a peasant levy (you can even get more than a typical levy would, because its on the defensive) and you'll have soldiers to fight with. Yeah, okay, they aren't exactly trained soldiers but there would be a good number of them.
Not to mention, the Drakken are assuming an easy fight. They're going to attack very, very aggressively; with the right moves, the main army could be fractured into smaller warbands and fought individually, and armies could be goaded into overextending or marching into bad terrain.
Also
 the Drakken may have maps but they don't know the land as well as the people who live there. Most Gems will know what is safe to eat and what isn't, while the Drakken wont all know that. They will know shortcuts, they will know where they can hide themselves, they'll know which areas they could lure the Drakken into for an ambush etc

Yeah, the Gems don't hold many cards and the deck is stacked heavily against them, but they have their own cards without having to making the Drakken into a race of total idiots who take 5 times longer to learn anything. Besides, wasn't the whole point is kind of yes, the Gems are pretty screwed if they stand alone?

IF they stand alone; you're assuming the brides are at their most useful for espionage, sabotage and assassination. While all are potentially useful (if used at the right moment, naturally) you're missing a very, very important function; Diplomacy. This is sort of medieval, feudal period is one where foreign policy is almost indistinguishable from a rulers personal desires and relations; If a Bride could convince her husband to hold back his forces or even outright switch sides, thats going to be far, far more useful.