How do you moderate private communication channels?

Question

I would like to implement a feature in a digital community that would allow users to send chats to other users. The target application is mobile phones, and the idea is that they would use these chat features to communicate privately (either group or pm) instead of using an external chat application like Touch or Kik (or whatever the craze is this week).

However, in order to do that, I would need to replicate a number of the features these other apps have, which in order to remain appealing would almost certainly have to include sending pictures, perhaps video. Even though the community is built up around a game, it would still require these social features to remain on the users' radar.

I certainly don't want my users to feel like they're spied upon. But in an age where kids are getting felony charges for sexting eachother, I certainly wouldn't want to get accused of facilitating child pornography because a couple of my underage users hooked-up. And honestly, considering I met my wife on an online game myself, I wouldn't necessarily want to discourage adult conversations over such a communications channel.

How can I balance legal issues and rule obedience with users' privacy over a 'private' communication channel between two users?

Answer 1

How do you moderate private communication channels?

Don't.

1. Add a Flag as Offensive button to the receiving user's interface, and when they press it, you let a moderator read that individual communication in order to decide what action to take.

2. Add a Flag as Spam button to the receiving user's interface, and when they press it, you let a moderator read it and the most recent messages by the sender in order to decide what action to take.

If you're worried about users sending inappropriate images, then don't allow them to upload the images to your site, this will mean the users will have to upload it on a public image hosting site which should have the software and resources to effectively moderate against illegal content.

Answer 2

You don't monitor private communications. One of the moderators of Ars Technica had a post about how Ars handled the public compilation of Edward Snowden's posts on their forums. He notes the readers reaction to how these posts (which were public) were handled:

The Ars community reaction was mixed, but there was a significant subset of users—many of which were folks who'd been posting at Ars for 10+ years—who felt shocked and betrayed that Ars would spotlight a poster's public posting history like that, even though none of the information was private in any sense of the word. [...] The general reaction to "exposing" a user's public data, even a user who was unquestionably an enormously important and newsworthy figure like Snowden, included a very large amount of community anger.

This clearly highlights the user's expectations, even for thier public posts. Private channels should remain private. If a user is being threatened, abused, stalked, etc. over a private channel you can have a "Report" button that forwards the communication to the moderators. The remainder of the messages remain private, but the communication thread that needs to be acted upon is now available for moderators to see. The user's expectation of privacy (regardless of whether you grant them that in your terms of service) remains intact, as they have taken the step to include you in this one instance that needs to be resolved.

The Ars moderator continues:

If Ars had grepped through the database to also look at his account's private messages, which we did not, that rage would have been far greater—and, in my opinion, it would be completely justified.

Ars had a story about a very public individual and likely has information this individual created on their system. Yet, they are unwilling to even look at that information for fear of violating their community's trust. This is a very powerful factor. If your community can't trust you, as the leadership of the community, there is no point in sticking around.

Answer 3

Do not monitor private communication channels. The users will expect that only the recipient and themselves can read the message, and nobody else. Attempting to monitor these communications is an invasion of privacy, and almost defeats the purpose of having a private communication channel.

I propose that you implement one of the following protocols in your system:

• Blocking. Give users the ability to stop others from communicating them. It may be wise to track how often a user is blocked, as this can be a sign that he is breaking the rules. Ignoring is another term that is very similar to this.

• Reporting. Billy may block/ignore Joe only due to personal preference, not due to a rule violation. However, to make it easier to track down rule breakers, implementing a reporting function will be helpful. The reported message can then be read by moderators, but only that message.

• Want to avoid inappropriate images and/or videos? The reporting feature will help, but the best way to avoid this is to not implement a feature for uploading videos and images. If you feel this is a necessary feature, perhaps make this something the user can unlock. (Similar to how moderation features work on SE)

These are common features that users know how to use, and understand that a reported message will be read by a moderator. Users expect privacy for these kinds of communications, and attempting to monitor them will backfire. The above features will allow you to keep private communications in line with the community guidelines.