On Stack Exchange, how do I motivate users to prioritize the needs of the community over personal gain?

Question

On Stack Exchange, users gain valuable Internet points (reputation) when their questions or answers are upvoted and lose (fewer) points when they're downvoted. This creates an incentive to try to answer any question where you can expect a net-positive change in score.

Some questions are really, clearly not good fits for the site -- they're off-topic, or they don't provide nearly enough detail, or they're otherwise pretty obviously on the road to being put on hold. Once a question is put on hold no new answers can be added until the question is edited (if it can be) and reopened. Questions are put on hold (and reopened) by user vote or by moderators.

On one site that I'm involved with, we have a widespread pattern of users rushing to answer questions that are, soon thereafter, put on hold. Sometimes these answers give good advice -- but it's advice about things that shouldn't have been asked on that site in the first place. Because it's good advice, it gets voted up even after the question is put on hold.

Sometimes several hours elapse between when such a question is asked and when it's put on hold, but I've seen this happen in under 15 minutes so I don't think "close faster" is an option. Our community is reasonably diligent here already, but also pretty popular, so stuff will get through.

Moderators (or, sometimes, users or automated processes) can delete these questions eventually, but I'm not looking for ways to clean it up -- I'm looking for a way to get users to not answer doomed questions in the first place. I realize it's their time to waste, but sometimes this situation creates drama when a question is deleted and people who answered it lose reputation points. Secondarily, it's harder to edit a question that's been put on hold to fix it if there are already answers; we try not to invalidate existing answers, though in some cases we do that anyway and edit the affected answers.

How can we get users to set aside their own possible reputation gain and instead vote to put on hold questions that do not fit in their current form? I am looking for a behavioral solution, not a technical one.

Answer 1

I realize it's their time to waste, but sometimes this situation creates drama when a question is deleted and people who answered it lose reputation points.

That's their fault. Make it clear on your site somewhere (meta, perhaps?) so you can reference it when they get mad and it gets deleted.

So:

• Make it clear somewhere that this behavior isn't allowed. You can't really punish them for this, but let them know that the rep will be lost when the question is deleted. When you post this, make sure the post gets a lot of screen time.
• For SE, downvote the question. It won't cost you anything, and it'll make it easier to get it auto-deleted to remove that rep.
• If you see a particular user routinely doing this, don't be afraid to tell them. No need to throw them under the bus, but just make a simple comment on that they'll probably lose that reputation.

---

<stack-exchangeism>... for the users who upvote, not post.

As always, votes are the property of the user so they can do whatever they want with them unless it is fraudulent. Also, new users may not know about if a question is on topic or not.

In fact, the tooltip of upvote says this:

This answer is useful.

People will upvote if they like the answer no matter if the question is on topic.

</stack-exchangeism>

Answer 2

The best bet is to remove the personal benefit to the bad behavior. If you reward people for a bad behavior, they will continue to do it even if it is bad. In order to prevent it, remove the incentive and, if necessary, add a penalty.

Removing point gain from a post that was very clearly not a good fit for the site removes the incentive to post since they realize there is a good chance they will just be wasting their time.

If the problem persists, then making it so that answers on downvoted questions would actually share the point loss of the downvotes on the question (if it is closed for being off-topic) might make sense to make it actually hurt the user to answer really off-topic questions.

Answer 3

Given your observations on answerers behavior, consider re-checking assumption about questions being "clearly", "pretty obviously", "doomed". Or more precisely, assumption that this understanding is firmly shared by site community.

I recommend to focus on this because in my experience gaining "the skill" to see that question is bad fit before it is closed always took me substantial time and effort. It never (never ever) came easy. Every time I learned to see how particular kind of questions is bad fit, it took quite some amount of studying of questions closed that way by others.

You may hope that clear FAQ / help pages help in there, but it wasn't so for me: it only helped to understand why question is closed after it's over (and this is important, don't get me wrong), but learning to recognize this before closure always took much more than that.

• ^{The way to check your assumption is probably to learn about how much negative feedback off-topic questions get before closure. Do questions get negative score, do users post criticizing comments etc etc. If you don't see that happening, assume the worst: people either disagree with your vision or don't share it firmly enough to act upon it. Negative feedback after closure wouldn't tell much, because after it happened and everyone can see it, it's really easy, "ohh I see it, it's in the FAQ, it's off-topic indeed, how could I miss that!"}

If you find that people can't see questions fate as clearly as you do, consider focusing your effort on getting them there. ...And don't expect it to be easy because if many of your users are like me, it ain't gonna be so, be prepared to struggle against a strong force.

I answered questions that I knew were clearly off-topic at least twice and I can tell you how it feels like. That sense that you can share some great insight and explain things clearly, it is really powerful and hard to resist; if my sense of topicality was weaker, I'd answer 20... 200... 2000 times, not just twice.

How to get there I can't tell, there probably can't be a guaranteed recipe. And it certainly depends on the size of the site. Say, at a site getting 10, 20, 50 questions a day, you can expect your commenting and voting on questions to make a noticeable difference, but if a site gets few thousands questions a day (like Stack Overflow), this will hardly make any noticeable impact.

Or, if there are like 10-20 active community members willing to learn and help to moderate, leading by example can be of great educational value, but if there are hundreds such users, you'd likely need more than just that.

One thing I can tell, again, from my own experience, is avoid pushing users to learn too much in one step. It never happened to me that I could grasp all the kinds of wrong topics simultaneously, it always was gradual, typically one topic a time. Only after I learn how to recognize particular kind of troublesome questions, only after that I can proceed to learning next kind.

Consider studying about your community learning pace and adapting to it. If needed, be prepared to even narrow down / modify your off-topic definitions. Say, you may note that community doesn't seem to share your vision of frobnication questions being off-topic but appears to be gaining consensus that some of these, about fungular frobnication are bad fit.

You are free to continue pushing users about general frobnication, but if you think of it, a more promising approach could be to first build and strengthen wide consensus about fungular kind and possibly proceed to wider kind later, building on prior effort and success.

• ^{For a more realistic example of topic adaptation, refer "Where to start" kind questions at Programmers Stack Exchange. Per se, these can be considered a subset of a more general too broad category of troublesome questions. But while too broad questions are generally treated rather mildly (for a good reason, but that's a different story), that specific subset has gained a strong and wide consensus of being bad fit. Where to start questions are often heavily and quickly voted down, criticized in comments, and... get answered quite rarely.}

Answer 4

From behavioural point of view, as long as the people are getting the positive feedback, they will continue their behaviour, with even increased strength.

They answer off-topic question, they get some reputation, they continue doing that in the future.

The best solution, in my opinion, is to eventually erase the off-topic questions together with the answers and the positive reputation for the people who have posted it. Then you have the negative feedback. The people finally understand, that answering such questions is a waste of time - they initially get some reputation, but eventually lose it. It's better to make more effort and answer better questions.

Delete only those questions which are literally off-topic, so that the answers are off-topic too (example: opinion-based and resource-request questions). If question is widely speaking on-topic, but "only" terribly written, it's still possible for it to get some great answers, so don't punish users for answering them.

Of course, don't expect immediate effect. It will depend on how quickly the questions will be deleted.

Answer 5

The only thing I would like to add to the other answers is that it seems for some reason that members of your site have a fundamental disagreement about what is on/off topic (if they thought it was off topic, they wouldn't be answering these questions).

Perhaps instead of unilaterally closing these questions, the moderators could engage these users (in chat or in meta) to discover why they think these questions are on topic. The topic could then be redefined with their feedback, or these users could be reeducated on what is the topic for the site.

Answer 6

Perhaps you should not be "looking for a way to get users to not answer doomed questions in the first place." Some of the most useful questions and answers on here are ones that I have seen that were ultimately closed.

Of course many are closed that should be closed, but some are closed due to some "technical violation" while they are still able to convey lots of useful information to users of the site.

I think that the points system is the technical solution, and it can be tweaked if really needed. I don't think that a behavioral solution really exists. It's just like some people will choose to do good and make $20K working at a non-profit, while most people will choose to do work and make $80K per year working for a corporation. It's unrealistic to expect people to do things against their own economic self interests. That applies if the economy is based on dollars or points.