I just took it as they was physically incapable of using other magics as the other magics just would be "No, I don't like you, you won't bend me to your will"