I get what you're saying. Freedom of speech is one thing, everyone has a right to their opinion no matter how misguided it is and it's part of humanity and living has a collective, but banning or making that group illegal would only turn them into extremists, and we sure as heck don't need more of those. 

From my experience, it's not by banning someone to do something that they'll magically change their mind and their ways. It's only by themselves that they can see how wrong they are and in most cases it takes a long time. There's also the whole pressure and brainwashing of an organized group that makes it harder for people to get out - just imagine how much worse it would be for them with the added pressure of their gatherings being criminal. Their leaders would have to be extremely strict.

Not to mention they'd be targets for other crime groups - which wouldn't be all that bad if you think about it. Racism is disgusting, but they have a right to redemption just like anyone.