13

If needed, refer these moderation strike announcements for more background: MSE post and open letter.

As one can see from above references, the relations between volunteer content curators and the company owning the site haven't been smooth. And this looks natural to me given the difference in their interests. Company interest is traffic and money. Curators interest is accurately rating site content (to prevent reader picturing them sinless angels, some openly admit doing this for selfish reasons of using the very content they curate).

So far it was possible to keep the relations despite tensions between curators and company because both sides made compromises (especially when pressed strong enough). I would even go as far as to say that in the root this relation was supposed to be symbiotic: quality of the content maintained by curators helped company get traffic and money while company money could be invested into improvements making it easier to maintain content.


Now, it looks like ChatGPT (and other similar tools) introduced a new dimension where their perceived interests became antagonistic so that wins for either of the sides would be interpreted as losses for the other. ChatGPT essentially took part of the questions that were historically asked at the site and began providing answers for these in smoother and more convenient way.

This tool is expected to provide satisfactory answers for duplicates - questions that were already answered elsewhere. From this point of view, it takes away traffic that shouldn't be there on the site to start with: instead of posting, askers would better search and find existing duplicates of the question they wanted to ask, along with the answer they are looking for. Considering that, from the perspective of content curators, ChatGPT can be helpful in that manner.

For curators, questions "intercepted" by ChatGPT mean just less load on maintaining and duplicating these questions if they were posted to the site and more time to spend on improving content quality and answering non-duplicate questions.

For the company though, ChatGPT presented a difficult challenge due to its negative impact on site traffic (which decreased 14% in March according to similarweb). It looks plausible that the tool may eventually get to the point of identifying most existing duplicates and presenting their answers with sufficient accuracy and this may drastically decrease amount of questions asked (N.B. ChatGPT is not yet providing 100% reliable answers).

As far as I can tell, company perceives this as a very serious danger to their business. And their actions related to this seem to indicate that they have chosen a way to address it that makes content curation impossible ("effectively permitting nearly all AI-generated answers to be freely posted" according to referred open letter).


My understanding of the company's plan based on above is, they are going to totally change how site works inside - specifically, remake it into a "ChatGPT frontend".

I think this is supposed to work as follows: imagine someone unaware of ChatGPT has a question. They get to the site historically known as the place to ask (that's Stack Overflow brand) and post their question expecting some expert to answer. At this point some user looking for rep points passes the question to ChatGPT and translates its response into the answer box.

Long term, above approach may be a dead end because it is so inferior to using ChatGPT directly. But short term, company probably believes that it will help recover traffic and this is maybe why they act the way they do and why their actions may look reasonable to them. It's hard to tell when critical mass of those attracted by the trick will realise that there is much more convenient way than roundtrip through the site and all this "artificial" traffic disappears.

From perspective of curators, company approach may look deeply (maybe even deadly) troublesome. The main problem is not really that it looks like a dead end - after all company already tried risky experiments that ended in tears and was able to recover (Documentation comes to mind as an example).

Worst thing about this approach is that - unlike previous dead projects - it appears to be very difficult (if possible at all) to recover after it's gone - because in the course of it most if not all users who maintained content quality will likely leave (quite possibly along with those who provided high quality human made answers, but I digress).

Really, while it looks doable (although not easy) to rate and curate answers provided by humans - doing this on AI generated content is totally different.

I don't think that those who maintaned human made content quality will have scale, motivation and abilities to suddenly turn into unpaid slaves used to rate automated tool output - especially given that as I wrote above most of this output will likely go to duplicate questions that should be closed, not answered.


OK so here we are...

  • If things work the way curators want, then company may expect fairly large decline in the traffic (one can argue that in theory this traffic shouldn't be there in the first place, or the decrease would not be as significant, but it's hard to convince company).

  • If things work the way company tries to approach their challenge then curators will probably have to stop what they were doing all these years and maybe even leave the site because there is no way to tell when this ends.

Given all of the above, I can not figure if there is a way to keep relationship between content curators and the company - ideally mutually acceptable, or at least with tolerable compromises for involved sides. Thoughts?

5
  • 2
    wonder if end-of-life tag applies here
    – gnat
    Jun 6, 2023 at 22:56
  • 1
    I don't agree that the decline would be as dramatic as you characterize it. Moreover, I struggled to follow your train of thought until I got to: "I think the way how they suppose this to work is as follows:..." You may consider make the previous paragraphs a tad shorter, in the interest of brevity. Just a suggestion obviously. Cheers.
    – M--
    Jun 7, 2023 at 0:07
  • @M-- agree regarding decline. I edited to clarify that this is merely a pessimistic expectation. Also you make a good point regrarding making that paragraph shorter, I need some time to figure how to cut it
    – gnat
    Jun 7, 2023 at 5:17
  • @M-- I dried this paragraph a bit. Though frankly it still feels a bit heavyweight, so if you figure how to squeeze it further, an edit would be appreciated
    – gnat
    Jun 7, 2023 at 11:23
  • 1
    That paragraph was fine, if you reread my comment, I said I had a hard time figuring out what is your intent until I got to that paragraph. And I said I would make the previous paragraphs shorter, so you get to the point faster. :)
    – M--
    Jun 7, 2023 at 13:24

2 Answers 2

4
+50

I am joining this site just to be able to reply to this post.

Premise:

I think this is supposed to work as follows: imagine someone unaware of ChatGPT has a question. They get to the site historically known as the place to ask (that's Stack Overflow brand) and post their question expecting some expert to answer. At this point some user looking for rep points passes the question to ChatGPT and translates its response into the answer box.

Long term, above approach may be a dead end because it is so inferior to using ChatGPT directly. But short term, company probably believes that it will help recover traffic and this is maybe why they act the way they do and why their actions may look reasonable to them. It's hard to tell when critical mass of those attracted by the trick will realise that there is much more convenient way than roundtrip through the site and all this "artificial" traffic disappears.

In my humble opinion you are missing a critical aspect of this comparison. the company (in this case Stack Exchange) has access to an unpaid workforce that can be herd to "fix" and "improve" those generated answer.

If we also consider this aspect the comparison does no longer hold. On one side ChatGPT (or any similar tool) can just provide an answer and when confronted for evidence or error will usually agree with the user even if the user is lying.
On the other hand a curated site like Stack Exchange will provide you a fast answer that will make the site look good (even if the answer has defects or is plain wrong) and then someone will do the work to savage the generated answer or post another one.

In the end the curated site wins because it is not just "generated content", it is "generated content curated by our trained monkeys".

So in my humble opinion having both the AI power for producing speedy "turds" (to quote shog9) and the monkeys to clean them up should look quite profitable and a great way to compete with tools that can only produce unpolished generated answers.

Conclusion

What I just said implies that there is no real answer to your question as both parties are playing "rational" but have different and almost opposite objectives: the only resolution can come from one of the two sides conceding and changing their view.

It is also clear that the userbase can't change theirs: it is the only reason they are curating content and giving out their time for free in the first place, so as soon as they are no longer satisfied with the site they will move out and the community will inevitably fall apart.

Again, in my humble opinion this means that any resolution can only come from a change in the company view that must realize that they can't pursue both objectives - unpaid content curation and AI generation - at the same time.

5
  • consideration of unpaid workforce is a good point to consider, I didn't include it in the question because it already felt hard to read having so many details. Your answer caught me in the middle of writing comment to another post addressing this concern. :) The assumption of free curation becomes slippery once we take into account that company expects ChatGPT stuff to offset fairly large amount of lost traffic. Because nobody in their sane mind would agree to mode/rate loads of automatically generated content for free...
    – gnat
    Jun 7, 2023 at 16:50
  • ...related reference is here. Although the topic looks different on a surface, substantial similarity is, it was an attempt to easily get lots of attractive content. To have a free lunch so to speak. "Tons and tons of people visited the site... but very few decided to stick around. Turns out that... very few are actually willing to spend any time to... maintain and moderate..." It is up to the company to disregard this prior attempt but it probably would be safer for them to estimate cost of hiring paid moderation team, for just in case
    – gnat
    Jun 7, 2023 at 17:03
  • @gnat Sadly I think that you assumption that "nobody in their sane mind would agree to mode/rate loads of automatically generated content for free" is also a mistake. Personally, I think there is plenty of users that are willing to do so. Otherwise, the strike would have a far more visible impact: instead, people continue to curate stuff under the claim to either wanting to avoid leaving bad content go rampant while still supporting the strike, supporting (openly or not) the company, being apathetic or even oblivious to what is happening.
    – SPArcheon
    Jun 8, 2023 at 6:55
  • @gnat Imho a strike is not a pacific act of resistance - a strike is an act that will cause damage precisely because that damage is the only way left to force-hand the company to listen after words failed, so I see this attempt to "We are on strike but we still want to curate" quite ironic, but that is not the point. The point is that even while striking there is plenty of users that are still wiling to polish turds for the company. You really think that they would go magically away even if the company didn't listen to a single word? The only thing that can move them now is image damage.
    – SPArcheon
    Jun 8, 2023 at 7:00
  • same assumption of free maintenance was made before NPR and we know that in the longer run it failed: "very few are actually willing to spend any time to... maintain and moderate". With GPT I'm even more skeptical because its perfecty correct grammar and syntax don't even make room for rep hunters willing to get +2 points with suggested edits. As for user's behavior, I wouldn't rely on projecting what we see now on the "GPT future". Currently site is mostly GPT-free thanks to prior moderation efforts and this makes many feel it's going as before - but it's not yet "properly flooded" by GPT
    – gnat
    Jun 8, 2023 at 9:38
1

... the relations between volunteer content curators and the company owning the site haven't been smooth. And this looks natural to me given the difference in their interests. Company interest is traffic and money. Curators interest is accurately rating site content ...

I am not sure if I see the situation as something bound to happen. A well-curated website is more desirable to users looking for answers (referred to as selfish reasons in your post), hence attracting more traffic and consequently generating more money for the company. I believe the company fails to see the big picture as I described it. While events such as introduction of GPTs can have and have had an adverse effect on the site's traffic, I cannot in good conscience agree that what company chose to do is the appropriate response even from their point of view. Preventing us from effectively moderating the sites will result in deteriorating quality. While it may cause the traffic to bounce back up in the short term, in the long run it will be catastrophic from anyone's perspective.

You said it yourself:

I would even go as far as to say that in the root this relation was supposed to be symbiotic: quality of the content maintained by curators helped company get traffic and money while company money could be invested into improvements making it easier to maintain content.

Can you elaborate more on why this is not true anymore? If we end up with a site that is heavily cluttered with AI generated content, why one would choose to come here instead of directly going to the source?


Given all of the above, I can not figure if there is a way to keep relationship between content curators and the company - ideally mutually acceptable, or at least with tolerable compromises for involved sides.

My two cents on this:

Company needs to realize that what they are planning is a short term "painkiller" which would cause experiencing a much grater "pain" in the future. Moving forward with their plan will result in a serious decline of the traffic eventually, even if moderators/curators were onboard to rate the AI generated content and make the site a front-end for GPTs as you suggested. Again, why would one settle for the second hand information when you can go to the source?! Let alone that insisting on their stance will probably drive many of the active moderators/curators away.

With the hype around "AI", whatever that means in this day and age, users will know about them more than they know of a network of sites. So, if the company wants to survive and flourish, they need to offer a different product, i.e. a collection of quality questions and answers that expands over time with the contribution of real people and is properly curated, instead of giving in and make the sites yet another "URL" for accessing "ChatGPT".

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.