Could be that the races that are left are descendants of these fallen civilizations.

Maybe these older races tried to overcome a magical/cultural [url=https://en.wikipedia.org/wiki/Technological_singularity#Non-AI_singularity]singularity event[/url] and failed?

Maybe they created some sort of magical plague or blight that destroyed them?
Maybe magic is uncontrollably rampant in some places and absent from others, and causes strange mutations and corruptions of normal creatures?