True... 
Unless he and Midori meet in the present and both of them travel back together 

Or the God who summons them can tell them there's someone they need to bring back from the present and that could be their first objective