15

On an SE site I moderate, there has been a lot of off-topic questions lately. Nearly 40% of the last 50 questions have had to be closed or migrated. This is a huge problem as almost everything else about the community is going well: good traffic, many valuable users, and solid answer quantity and quality. The only problem is that are quantity and quality of questions is bad. Right now, I am mostly worried about the quality of the questions.

We have some very good questions that help keep our user base interested. Then there are the some average questions. Then there are the questions that are only asked because new users tend to think our site is a dumping ground for other questions. I know that new users get pop-ups that tell them how to ask, but I doubt that many of the read it. I also feel that some of them have been driven away after getting a question or two of theirs away, so I also don't want to drive away potentially productive users.

How can I make sure that new users know the scope of my site before they ask their first question? Is this even possible?


Note: The site is Lifehacks.SE

  • 1
    sorry, I'm new here, but shouldn't this question be asked on the Meta SE? – Ooker Apr 28 '15 at 17:56
16

What do new visitors see?

Usually, most off-topic questions are asked by new users who aren't familiar with the site. If this isn't the case, your community has deep problems with disagreement on the scope, which I'll touch on later.

Lifehacks.SE is a community with a broad appeal:

  • The subject matter doesn't require any specific competence — anyone can be interested in life hacks. (As opposed to a community about, say, bovine dentistry or 1970s video games, which cater to experts or enthusiasts of a particular topic.)
  • The form of participation has a low threshold — you just need to formulate a question about the topic. (As opposed to, say, a wiki, where people will participate if they feel they have something to contribute, not merely something to get.)

In such a community, visitors are likely to have had no prior contact with your site. Most people don't take time to read all the documentation that's available to them. This is what your visitors see:

Lifehacks screenshot

A few users will take the time to click the big shiny button “Take the 2-minute tour”. A select few will even take the time to read the tour page. Most, however, will jump straight ahead to what they wanted to do in the first place, namely, “Ask Question”.

The fact of life is that for the majority of visitors, what describes your site is its name. And that's if you're lucky: on Computer Science, we get quite a few “fix my Windows” questions, from people who stopped reading at the first word. (And then there are the desperate “hey, a text box” cases, who ask about gardening or relationship advice on Stack Overflow.) The better ones will also read that introductory sentence (“is a question and answer site for people looking to bypass life's everyday problems with simple tricks”) and at least part of the tour page.

Levels of scope description

There are a number of successive levels of precision when describing a Stack Exchange site (and you'll find similar patterns with other types of sites):

  1. The site name.
  2. The one-sentence description at the top.
  3. The list of topics on the tour page. (There's also a similar page in the help system, but only Stack Exchange old-timers know about it; it's irrelevant for visitors who are new to Stack Exchange, but useful for veterans of other Stack Exchange sites.)
  4. Policies described in the most visible questions on the meta site — conventionally with the [faq] tag on meta.
  5. The information that can be found in a search on the meta site.
  6. The habits of the community, which may or may not be fully documented on meta.

New visitors only see the first three, so the last three aren't relevant for them. The last three are relevant for returning visitors and regulars of the community. It is however important to make them all consistent. Each level should be a summary of the next level, adjusted for the available space. In order to provide the right first impression and the right advice to visitors, you need to work from what is actually there on the site, make sure that all important things are discussed on meta, gather policies in a few threads.

That, by the way, is a downside of having sites named after their topic. A site name like Super User doesn't make the scope immediately clear, which pushes people to read at least the one-sentence description (“a question and answer site for computer enthusiasts and power users”). Counter-example: Theoretical Computer Science — it's a “question and answer site for theoretical computer scientists and researchers in related fields”, but it's often hard to convince people that no, questions that are not research-level are not accepted. With a site name like “Lifehacks”, there's no incentive for new visitors to explore the scope in more detail, it seems obvious enough.

In addition to the levels above, it is useful to have an elevator pitch for the site (something that Stack Exchange founder Joel Spolsky is keen on, rightfully so). This is a little longer than one sentence, on par with level 3 above, but with a different setting in mind: the tour page is intended for people who are already visiting the site and about to ask a question, whereas the elevator pitch is more intended for people who aren't in front of a computer, but might be interested in your site. The elevator pitch is a very short introduction to the site with no supporting material. Can you summarize what your site is about in a minute or so?

Converging on a scope

If new visitors are to have any hope of roughly understanding what the site is about, it's vital that the regulars are in broad agreement, if not on what should be in scope, then at least on what is in scope.

There can of course be debate about specific matters. Not every debate has to be settled here and now, it's ok to leave minor issues open — if a type of question comes around one a year, it doesn't matter if you're not sure what to do with it the second time round. Major issues — the kind that crop up every few days — need to be settled. Consensus is best if it can be achieved; when there's a debate with two or more sides, try to find a compromise position that everybody can be persuaded to agree on. But if consensus can't be achieved, take a majority vote on meta, or have the moderators settle the discussion.

When discussing scope issues, keep in mind that your discussions are not what new visitors will see. Avoid having a scope that is too complex to describe: it's an exercise in frustration as you explain that despite the title, the site is actually about… Scope determination works in both directions: sometimes the scope is whatever the engaged community wants it to be, but sometimes the scope should really be what it appears to be at first glance. As a moderator, you're expected to be a community leader and remind people of such considerations. This doesn't mean that anything should go: if you've determined that a particular type of questions is problematic, then by all means exclude it. But don't go gerrymandering — don't exclude subtopics for reasons that are external to your site.

If the boundaries of the scope are subjective, they're harder to explain. This is a problem that Lifehacks.SE faces head on. Lifehacks.SE does not have a clear scope to regulars. Early on there were many, many discussions on the broad strokes of the scope, and it seems to me that they're petered out not so much because the question is settled as because participants have become bored with them. The most official summaries of the scope of Lifehacks.SE are:

Unfortunately, the “manifesto” — like many other meta discussions — draws a subjective line between:

  • “uncommon solutions to common problems”, “unusual ways of using everyday objects”, “problems that may not have an obvious solution” — on-topic;
  • “conventional ‘how to…’ questions”, “using products in the way they were designed to be used” — off-topic.

The problem with these one-sentence descriptions is that they don't really tell much. Where is the line drawn between “conventional” and “uncommon”/“unusual”/“not obvious”? This is a subjective judgement. The regulars often don't agree (there have been many meta threads on that topic), so you can't expect new visitors to have a clue. Everybody draws the line in a different place.

This is compounded by the fact that the line is only important to enthusiasts, not to casual participants. Life hack enthusiasts are bored by the conventional. But casual participants don't care: they have a problem to solve, never mind whether the solution is actually unconventional. This is rather problematic on Stack Exchange, which is focused on questions where someone has a problem to solve.

Broken windows

I wrote above that what new visitors see is the front page. But, as the site gets more popular, this will become less true. More and more, people will discover the site because they found a question, or more likely an answer there. The ones who stick might go and browse a few other questions.

It's important that the questions and the answers that people see reflect what you want the site to be about. To that effect:

  • Make sure to close questions that are unsuitable. Don't leave a question open just because it's popular — if it isn't the kind of question you want to see again, close it.
  • Many questions from new visitors are unsuitable as originally asked. Keep in mind that you can always edit questions. It's better to have a slightly different, on-topic, open question than an off-topic, close (or worse, open) question.
  • Whether or not the question can be edited to match the scope, be sure to explain why the question was closed. It often helps to cite a meta discussion or site policy that clearly applies to the question. Avoid community-specific jargon like “shopping question” or “mind hack” that your target audience won't understand.
  • Delete closed questions if it's clear that they won't be reopened. Casual visitors don't pay attention to whether a question is closed. Not only that, but if they aren't Stack Exchange regulars, they probably don't understand that “closed” means “undesirable” — on many forums, “closed” means “old” or “solved”.
  • The same goes for answers to some extent (less so, because fewer users will base their interactions on the answers they've already seen, but it still matters). Having good answers is what will attract visitors in the first place, but not having bad answers is important in getting them to post good answers.

Conclusion

There is an unavoidable influx of off-topic questions: you can't expect everybody to get it right. But you can keep it to a minimum if your scope is obvious. Failing that, at least arrange to have a scope that is clearly explained.

Lifehacks.SE fails on both counts, which explains a rather high rate of closed questions (12 of the 50 most recent questions as I write — compare with 7 on Computer Science (mostly “I'm a CS student so my programming assignments must be on-topic”); 13 on Skeptics which has a non-obvious requirement that all questions must take the form of a notable claim).

To reduce the rate of closed questions, the scope needs to be made simpler. It also needs to be made more relevant — as I explain above, conventionality is not something most askers care about. There is a choice to be made here: if you make the scope more relevant to new visitors, you'll make it less relevant to some of the regulars, which is bad. You need to find an acceptable compromise.

  • 4
    Well-done! This is a really great example of answering both the specific and the general; it's not just about LifeHacks but it uses LifeHacks as a good example of the points you're making. – Monica Cellio Apr 27 '15 at 23:51
9

Disclaimer: I've poked through Lifehacks Stack Exchange a few times before. I'm going to try to keep this as generalized as I can, but I may point out a few trends that I've noticed specific to LH.SE.

Whenever there is a problem like this in a Stack Exchange community, one of these things is almost always the problem: site scope not clear, site scope not read by new users, no 'conversions,' and bad traffic.

Scope Not Clear: There is always a bit of site scope issues with a new SE community (Community Building has had our fair share) but there gets to be a point when the scope issue is too broad.

It seems like your scope is incredibly clear from meta ("out of the box thinking"), but not everyone agrees on how to execute this scope. For example, there are 2K users who have their questions closed. Sure, things happen, but the first step is to make sure all of your top users are on the same page and everything is laid out on meta clearly. It seems as you've already done this, but it doesn't hurt to be sure that everyone knows about this.

Scope Not Clear to New Users: There isn't much you can do about this. Stack Exchange doesn't provide a lot of tools to help with this issue. It seems like you just have to close questions and leave quality comments.

Convert Bad Questions to Good Questions: Speaking of quality comments, I've found that, with all moderation, the best comments are in a four part response:

  • Some sort of greeting
  • What they did wrong
  • How they can fix it
  • Some sort of "thanks"

For example:

Welcome to Lifehacks Stack Exchange! For a question to be on topic here at Lifehacks, it must be asking for an out of the box solution, not just a simple how-to. For more information on what's on topic, please see this post. Please use the edit button to edit your question to include more details of why the current solutions aren't ideal and what you expect in a good solution. Thanks!

This is friendly, it tells them what's wrong, it tells them how to fix it (turning a bad question into a good one). Furthermore, it gives them links that they can look at to further their understanding. I've noticed that people are more likely to read links after their question has been closed, so it's especially important to educate the OP and give them an easy path for reopening the question.

Currently, most of your closing comments don't have this call to action and/or they don't explain what's wrong in the first place. Telling them it's off topic is fine, but they'll never learn and are going probably walk away if you don't show them how to fix their question.

You need to educate your community on this to try to get some questions reopened by an edit from the OP (feel free to use this post as a base for the meta post). Even if a question seems unsalvageable, it's still a good idea to let them know how they can improve their question to be courteous.

This Chrome plugin will be especially useful for having these high quality comments already typed out.

Analyzing Traffic for Bad Advertisement: One last thing that you should do as a mod is to use the moderator analytics tools. Try to identify if there are any sources of traffic that generate low quality posts. For example, an advertisement on a general forum might not be the best idea if everyone there is looking for inside the box answers on that forum. Try to get any negative traffic removed if you can. This approach is by far the hardest to execute with limited data, but it can also be highly effective if you can draw conclusions.

  • 3
    I'm surprised that you consider the scope “incredibly clear from meta”. In practice, everyone has a different opinion of what constitutes “out of the box thinking”, which makes question closure rather haphazard. Explanations vary from one meta thread to the next. Having participated on Meta.LH, and the close vote privilege, I still often find it unclear whether a question should be on-topic according to the official scope. – Gilles Apr 27 '15 at 23:41
  • @Gilles read the second part of that sentence :) "It seems like your scope is incredibly clear from meta ("out of the box thinking"), but not everyone agrees on how to execute this scope." I guess I didn't focus on that point of making sure everyone knows how to execute the scope as much as some of the other aspects. – Anonymous Penguin Apr 27 '15 at 23:59

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.