By justification for magic, you must mean stuff that makes sense in the context of their universe but not real world physics, which would suck all of the fun out magic battles being that they can't use magic in real life. At some point we have to make something up even if it mostly uses real physics or else it's not magic.

In Naruto, it's completely fine if they move their hands in certain poses and use their chakra to make a giant dragon made out of water pop out. Using this same technique to stop time needs no more justification than any other jutsu out there such as the water dragon example. All they have to do is be consistent with how they're achieving it and have in universe laws of magic even if physically doesn't make much sense in our universe's physics. I assume that's what you're saying, but I'm slightly confused then why you say nobody can explain how time magic works, so maybe I'm missing something. In this Naruto case it would work by doing hand signs and chakra just like any other jutsu. That explains it in their universe.