[@Prints Avoid] Maybe I could add a blurb involving the incident with his father and the Ministry's direct involvement? They could be worried that the boy would turn into a Dark wizard after such a violent first act of magic and wished him to attend Hogwarts so that he could be closely monitored. Does that sound a bit more feasible? 

I hate any kind of plot hole too, I was pretty tired last night and then the coding just made me frustrated lol