Seems like it's bugged just a bit. If you try to upload a new avatar directly over your current one, it doesn't replace it. However, when you click Remove Avatar, it will change. 

I expect that clicking Remove Avatar will change it whether or not you click the button before or after upload, as it currently works. It feels a bit unintuitive, but I think that's just a result of the primitive avatar system we have right now.