I think the toughest opponent to fight would be an intelligent, quick fighter in a good suit of armor and magic that can be used for a variety of purposes such as controlling energy to form swords, walls, beams, etc. Even in the armor they'd be just fast enough to give plausible deniability for being able to dodge you and just good enough armor to do the same but for defense and say they didn't receive too much damage, if at all. Their physical strength wouldn't matter much because their energy is their primary attack and can cut through defenses quite well.