The Thalmor also love taking credit for things they never had a hand in. So them going "this is elven ruins so step off" isn't too out of character, IMO.