I think you mistake my meaning. By taken away I don't mean the quirks, I mean the people are hauled off to the tower, as I suggested we play these people hardly knowing why we're taken in the first place.
Well if you wanted to widen the pool they at times 'take some' of the Discardables. Considering the elite never really cares much for the lives of those who are not of their concern. 
Think Roman if you will with slavery. People could be captured or even sold into slavery.