I am not entirely sure, but I think he did say that the High Elves were descended from the Alyieds. I only ever did the quest line once (Think I remembered his name too, Umbacano?) because the sod makes you go on a massive dungeon crawl for some god forsaken ugly alyeid statues before he gives you a string of quests which culminate in him turning into an alyeid lich king and trying to brutally murder you. Such a great reward for my service eh?
When you see the two Alyeid warriors in the thieves guild questline, they basically look like massive High Elves, so I think the Alyied were the ancestors of the high elves and the Alyieds had magic, pretty much built the IC etc. So the high elven claims wouldn't be strictly true
I like how the Orcs are elves, when I first played TES I didn't see that coming until I noticed that only the elves had Mer in their names xD