I wonder if sirens/succubi have other magic, for example: luring people away with magic/enchantments, taking the ones they want, and then disappearing. All that's left in the traveler's memories (the ones left behind) are memories/visions of will-o'-the-wisps. So they entirely forget about the presence/existence of the siren/succubus.

I saw this image... and for some reason I thought you'd like it Gonzo.
[hider=succubus][img=http://shadowness.com/file/item8/226100/image.jpg][/hider]