That's 4 seconds out of 60, and as telegraphable as it is, that's only going to be an issue if you're trying to do mage assassin. Get into a thick of combat, with the clashing of steel and explosions happening everywhere, and 4 seconds and a portal sound is just going to fall into the background. Of course there's no denying a flaming dagger in your back is going to leave a lasting memory. Also I was referring to your comment about how others have a 30 second cool down while it looks like most of yours have minute long ones. And while I did mention I wanted to see some cool downs, I didn't specify the actual length. I'd be fine if you want to shorten the length of your Fiery Sword cool down to thirty seconds, though the portal shall remain the same. 

[@deadpixel101] And yes I do. I was actually looking for that exact manga for this exact character, mostly cause I have a hard time finding pudgy guys in anime that aren't horrendously gonk or women.