I know my gem doesn't wear armor as portals are kind of armor enough. Plus it wont carry with her if she blinks while her robes do. 

Also Peridot kinda wore armor, didn't she? That robot suit she was in and all that.

[@Pathfinder]