So if I got this correct. Making a fist fighter type character is going to be gimped cuz the red magic slows you down.

Yes?

If so, then it's majorly inconveniencing but I could work around it for a gimmick.