Is the idea of crowd-sourcing moderation duties a reason to prevent automated processes from helping?

Question

There was a discussion yesterday on Stack Exchange that ended with the following:

[I]sn't the point of crowd sourcing to get humans doing things? Inviting AIs (or facsimiles thereof) to do the job seems to counter the system's design.

Users/moderators of various communities, is this your experience? Does the large scale crowd sourcing negate the need for processes that can automatically clean up bad posts?

I found this an interesting discussion because my last post here was about an overwhelming amount of flags I was seeing and I've been looking into ways to automate handling some of them.

Can (most?) automated processes be removed when a community is large enough so that the crowd sourced aspect of moderation can manage quality on the site?

Answer 1

Disclaimer: I have an answer on the linked question. I also run an automated process that at least partially prompted the comment that led to this question

---

I do not believe the question posed regarding AIs or facsimiles was questioning whether all automated processes can be removed. Doing so would open a site to untold amounts of spam or abuse. I think the point the asker was making is whether these automated processes can (or should) be used for all areas of moderating a community and whether these automated systems should be run by end users.

System maintained automation

It'd be hard to argue that some kind of automated protection isn't needed on the internet today. Stack Exchange runs a spam prevention automated process. It is very good at it's job. Without this process in place, there would be thousands of spam posts a day across the network. Outside of Stack Exchange, there are other spam prevention services. These systems are all configured by the community running the platform. It is inherently "part of the system". Users (usually) know that the protection is there. They see the benefit by the lack of gibberish that appears in their community.

There are processes run by Stack Exchange that determine the quality of a post. Again, these automated processes are "part of the system". The processes feed into queue that a human looks at to decide whether the post is appropriate for the site or not.

User run processes

Certain systems allow interaction with the platform via a programming interface (web API, integrated SDK, etc). With this a user (or group of users) could build an automated system to do "something" with or to the platform. Can these processes replace the crowd? Sure. Can they do it in such a way that the crowd still matters? Sure...with more work on the part of the process developers.

Bots remove human variance. If you had 50 copies of the same bot software mowing through the review queue, you might as well just replace the whole review process with a single bot running at SE HQ. - Oli

A bot is not infallible. A simple programming mistake can quickly cause problems. A bad training set thrown at a machine can cause problems. But, humans are also infallible. Gamification causes people to do weird things to increase a number next to their name. Checks/audits/human confirmation of bot activity is important.

Site policies are also important. If a bot is found to be misbehaving, the community needs a way to handle both the bot (possibly with a suspension) and the bot owner (if they are known).

Usage of automated processes

User built automated processed exist around Stack Exchange. I mentioned above that there are system maintained processes for both spam and low quality detection. The system, though, needs to be much more conservative in what it calls spam or low quality. The site is global and English isn't everyone's first language. There are also those more sophisticated operations that can evade the system attempts to stop spam. Users have stepped up and built systems that try to combat this problem.

• SmokeDetector: Monitors the real time questions feed and checks for black listed terms. These terms have been added by users across the network that have noticed patterns of spam or low quality posts.
• Pham: Another system built to monitor the network feed for low quality and spam patterns.
• Zephyr: A system built to monitor network chatrooms for mentions of low quality or spam posts. (Disclaimer: Zephyr is mine, and is separate from the comment flagging process). It can quickly show top spam URLs that have been recently seen which, in turn, can feed the other two bots with important data.
• Caprica: A chat bot (released on Stack Apps) built to help new (and existing) users in chat rooms (among other things)

Outside of Stack Exchange, there are other systems that can help moderation/site protection duties as well. AutoModerator is the one I am most familiar with.

Summary

The long winded explanation above is so that I can answer your question

Can (most?) automated processes be removed when a community is large enough so that the crowd sourced aspect of moderation can manage quality on the site?

I believe the answer to this is "No". But, those automated processes can help manage quality. Crowd sourcing is great because the duty of keeping the site clean is spread across hundreds or thousands of people on large sites. But, even on large sites like Stack Overflow or reddit, the crowd can't hold back the tide of internet sludge alone. Automated processes (both system maintained and user maintained) help to keep the trolls, spam and general low quality stuff away. When those processes can feed into areas where the crowd can help, it reduces the time needed to search for bad stuff. Instead, the systems flag it is possibly bad and a human wanders along to say "Yup, that's bad" or "No, that's good. Keep it."

Answer 2

Does the large scale crowd sourcing negate the need for processes that can automatically clean up bad posts?

No. I'm making a case for the opposite: you should use automatic systems where you can. It's a matter of cases handled correctly versus time invested in developing the system.

For any moderation item (I'm giving a binary yes/no example here, e.g. Is this spam?), make a 2x2 grid with on one axis "it is (yes/no)" and on the other axis "automatic system detects it as (yes/no)".

Tweak your automatic system so that it works optimally in either of the two 'Correct detection' quadrants. You should err on the safe side (prevent missed detections and false positives) when tweaking the algorithm. You can assign actual numbers (percentages) to the quadrants when testing your algorithms.
Then, use your automated system to handle those two quadrants and let humans handle the other two. Even if your algorithms have low efficiency (again: better low efficiency than errors), they can handle part of the work and remove the repetitive (obvious) cases from the human workflow. Your humans will appreciate having to handle those; they don't want the boring stuff (other than maybe for gaming, i.e. quickly getting high scores).