It is a bad idea to fall back on the "We reserve the right to (...)" phrase that administrators tend to use. This should be a very last resort, when your reason is absolutely confidential, and there is no way to explain it correctly. So while Madara Uchiha's answer is useful for some cases, usually you will want to tackle this issue with more tact.
There are several "excuses" that you can use for the user's rejection, which are much more honest and to-the-point, and I think this one is the best solution with the kind of person that you are describing:
"We have reason to believe that this person's personality would clash with the other team members."
This is something you can use for the "popular yet bothersome" kind of person. If they're popular, your users will know that they have a knack for being involved with drama. So even if you don't literally mention the drama, almost all users will understand what you are actually getting at. Users who don't know this particular person, will see this as a perfectly acceptable reason.