As far as I can tell, specific summons are totally cool. However, you can have two types of summons as the OP said (since you're a secondary summoner). So it might be better to go with a general type of summons. It's totally up to you though, if that's the case.

Also, the Guardians are a Wanderer group, not anti-Wanderer.