When Jack first met Nine and Rose during the Blitz he remarks: "I had two years of my life wiped out by the Time Agency. For all I know your friend is right to distrust me." So that's way before he gets recruited by Torchwood, and that's perfectly fine if you haven't watched that side show. (Although it does a pretty good job of explaining how Jack coped with being stuck on Earth) it was just an idea I had when I read that the companions of today could meet a past version and I think that after the events of the 1996 movie would do just fine cause I haven't heard his drama audios neither [although I should. Sounds like fun to me :3]