I simply don't have the time to build languages, so I tend to just lump them all into different dialects of "common", even if that might not necessarily be accurate given the amount of years that's passed since the beginning of the world I reference (the one in my head :p).

I think it's purely a matter of simplicity, though I don't think language names necessarily have to be linked to geography - especially when folks in a region operate on their racial identity. Though, certainly, languages do change given the slang in any given region that has even a small split from its neighbors, and certainly has different development than a land hundreds or thousands of miles away.