I understand that argument and will concede that point.  However, time manipulation is still free GMing.  Whenever it's used, there's nothing you can do and you have to take everything that is done because there's no way that your character can react to said actions.