All I am saying is even in heavy magic based things like DnD nothing is resistant to ALL magic, some creatures can't be charmed or can't be hurt with fire or other types of damage. 

So assuming your character's people are not affected by magic, period, it would be a matter of balance that they could also not USE magic period. 

Or that, like some creatures in DnD or other magic based games certain magics don't work but others do. 

Just some suggestions is all.