If you are going to argue that the [i]actual[/i] modern incarnation of Nazism should be removed from free speech, you likewise can say goodbye to the "democratic socialist" movement, the anarchist movement, the Communist movement, the X movement, Y movement, et cetera, et cetera and so on. Name any movement that is shaded any color of "bad", which is to say [i]all of them[/i], then attempt to pick and choose who violates what and why others are exempt. The short of this is that such a philosophy does not work, not at all in fact, and unquestionably by its very nature is contrarian to American belief. The Nazi ideology is incompatible with what the United States stands for - the same could be said for communism, of which has done as many and if not outright more horrific things. They are both masses of terrible ideologies, but until their words become actions, they are more or less exercising their free speech. 

If you start picking apart free speech and choosing what you disagree or agree with and outlawing whatever does not fit your personal brand of alignment then no longer is it free speech. This is part of the reason people so disdain the "free speech is hate speech" argument; there is no such thing as "hate speech" because it is exclusively subject to opinion. There are an innumerable amount of examples that further prove there is no semblance of balance.

"Pigs in a blanket, fry'em like bacon." or "What do we want? (Dead cops.) When do we want them? (Now!)"

Absolutely reprehensible and disgraceful... still free speech.

You cannot have both.