He means you don't have to worry about them...yet. :brow 

Also doing nothing to rain in your allies actions is just like watching a bully punch down a fellow student, your lack of action is action and speaks volumes. :P