It's implied he gave it back, despite the rudeness of others.

In other words, it worked in my head because I was up late.