The idea is the servants go in first, clear out the more physical obstacles, Monica, and other illusionists/skilled masters dispels them, we move forward. The Greys are directly in the middle of a circular forest. There is no going around them, and they're purposely holding themselves shut up in their own territory. They're forcing us to attack them in their territory in order for this war to actually continue. The enemy servants uses the traps, and defensive terrain to pick their fights. Some of the aggressive ones may make bolder moves. 

The true danger lies in the servants being backed with fake images, and hampered terrain. They are also effected by it but know better because they're aware of what is where.