Don't worry Athanshadow... We're all willing to help you figure such out.

Rather than him having been abused... how about his father being a bit of a duality? Both beating and loving his mother, but never even considering abusing the son? Its not at all impossible for someone to abuse someone they love deeply.

As for limits to weaken the relic:
The size of animal thing would be a good one. Controlling a couple or rabbits should be far easier than say, controlling a beast the size of a dragon.

You could also play with the idea of herd animals being slightly easier than pack animals, which in turn is easier than lone animals. And perhaps carnivores being harder than herbivores. directing a herd of cattle might be possible despite the large numbers, whereas controlling a pack of wolves can be somewhat of a challenge. controlling more than one bird of prey (like an eagle) might be almost impossible.

Also, in time you could have his ability to control/influence animals improve.

Another flaw you could experiment with is focus. If he's focusing on the animals, he probably can't focus as well on himself. The more strenuous the controlling is, the less he can focus on his own acts. Controlling too many could perhaps trigger unconsciousness? And he'd probably need protection when he's directing animals. especially if its in a combat situation.