That does seem a bit invasive to them, I mean, they have mining stations on the large asteroids and the planetoids in the system's Kuiper belt. They won't attack your fleet, but they won't see you with very good eyes.