Just to clarify, the Ragged Brotherhood was there to unite the members of the island under the Ragged Brotherhood and take the lacrima. Also there were only 8 Ragged Brotherhood members on the island including Aeidus and Cordellia so it's not like an all out war.