The issue is more on adding users than removing them. Though if a user doesn't play well with others, it would be a good feature to be able to remove/report them (I know the latter is something that's in the works).  

I've fiddled with PMs and can't seem to find any way to edit a permissions list.  Which would require making an new private topic any time there's a new user that should be involved (e.g. adding a new user to an RP PM for OC/IC purposes).  

And if it's desired that everyone have access to all the previous, private posts (e.g. a private IC thread that a new user should have a chance at catching up on), it would require archiving all the threads in hiders for every new thread.