Naw, don't worry, that's what the gauntlets are for! Really, there are only two ways for your character to die here:

1. Be an idiot and take of the guantlets, then challenge someone to a fight butt-naked or;
2. Be a bigger idiot and take of the gauntlets then shoot yourself in the face with your own magic just to see if you'll survive it.

Those two asides, the worst that can happen is some trauma and an aversion to big-ass fights :D