Of course you have the right to ask me to remove Bob. You don't have the right to force me, but it'd only makes sense that if he impedes your ability to have fun in any way, you should tell me so I can fix it. If you didn't, that would be silly. Having fun, after all, is the point of roleplaying.