When should a new site tackle its tag set?

Question

I've participated in several new Stack Exchange (beta) sites, and they've all had this same problem (which is again current): developing the tag set. I'm talking here about tags that don't fit neatly into ontologies (like location tags). I'm talking about all the other ones, the tags that describe the content in a less-structured way.

These less-structured tags are emergent -- several questions get asked that have a theme in common, and somebody creates the tag and applies it to them. Sometimes the author of a question creates a new tag as part of asking the question. Sometimes there's duplication, sometimes there's odd overlap, and sometimes the vocabulary choices that are clear to one user are puzzling to another. The progression I'm used to seeing goes like this:

1. Chaos: tagging is very haphazard and disorganized.
2. First refinement, maybe with discussion: one or more users try to clean things up a little to lay a good foundation.
3. More questions come in and flaws become more obvious.
4. Second refinement, with discussion: somebody organizes the Great Tag Cleanup on meta.
5. Tags limp along for a while, sometimes oscillating (use X / no this should be Y / eh maybe X was a better idea). Meanwhile, new questions come in, some of which don't fit any tags. (For example, this question doesn't seem to fit well with any of our tags.)
6. Ongoing "spot" improvements -- breaking up that tag that's on 40% of the questions (as it turned out), create those synonyms, retag those, etc. User interest and results vary. Retagging questions can be disruptive (those edits bump posts), so there's a tradeoff between messing up the front page and fixing things, with uncertainty about how much of a fix it is.

I'm participating now on a beta that currently has fewer than 200 questions. On the one hand, if we can establish some good patterns now, it feels like it should be easier to have good tags as we progress. On the other hand, if we wait until we accumulate more questions, maybe we can skip some of the oscillation -- but we make more chaos in the meantime.

How do we know when it's time to tackle topic tags? Are there rules of thumb?

I'm specifically asking about a Stack Exchange site, but other multi-user platforms use tagging, so I welcome insight from elsewhere too.

Answer 1

Start as early as possible. And continue doing it until a rational, coherent system emerges. Eliminate bogus tags before they become a bad meme. Don't let up until the regular users in the community "get" it, and are able to help maintain good tagging practices.

Because tag edits will bump posts, giant cleanup efforts are necessarily disruptive and unpleasant. The earlier you can get the tags right, the better. Particularly during the beta period, when there are few questions and many active users, the churn from retagging is harmless, and perhaps even beneficial.

From my experience as an early participant in vi.stackexchange.com, I would say that there were some important tagging decisions we got right:

• Banishing incorrect terminology. For example, many users try to use replace for questions about search-and-replace operations. No! In vi terminology, "replace" means overwrite mode (as opposed to insert mode); the correct terminology is substitute.
• Establishing conventions for tag clusters. For example, we used plugin-xyz for all questions about plugin xyz, vimscript-xyz for questions about writing scripts using xyz programming language, and filetype-xyz for questions about *.xyz files.
• Be vigilant about terms with multiple distinct meanings, and disambiguate them early. For example, does windows refer to questions about running vi on Microsoft Windows, or viewing multiple documents simultaneously within Vim? If you don't split that tag up into microsoft-windows and vim-windows, you'll end up with a tag cleanup effort. (Merging tags is easy. Splitting a tag is hard work!)

The naming of specific tags is relatively unimportant. Mass renaming is easily done by moderators in Stack Exchange, and synonyms can be made. There is not much to be gained by debating normal-mode vs. command-mode (which mean the same thing).

The richer the tag vocabulary that you seed the site with, the less likely the later users will feel compelled to introduce their own bogus tags.

Most importantly, the Broken Windows theory applies: if users perceive that the tagging system for the site is a hopeless mess, then they will not bother to tag the questions carefully, leading to a worsening mess.