15

So I am managing a lot of sub communities and it became very hard to keep track of every single one. Moderator features are still in development and it happened already that I didn't notice bad developments in some sections before it was too late.

So far I focus a lot on simple post and replies numbers. Furthermore we track subscribtions to those communities and the votes. So far so good but I encountered some problems with that: First of all it is very hard to see where problems occur at first glance.

When researching the topic I stumbled on the Community Health Index, which seemed quite thought through and I really like the approach. Basically they measure

  • Members
  • Traffic
  • Content utility (a function of the number of posts and the views)
  • Responsiveness (how fast does a post get interactions)
  • Interactions (unique visitor participators and comments)
  • Liveliness (how much activity)

and pack it into a nice function where you can see how the community is doing and where its problems lie.

What is missing for me is a way to measure the negative occurences such as reports, downvotes, people dropping out (mostly because of some pretty negative experience). The health index and its function is pretty good and thought through in my opinion but it isn't measuring directly the negative stuff. You can see the impact of it but cannot intervene in a proper amount of time. As we are growing this is becoming a daily struggle for me and I pray for the day we can elect moderators.

Until that, what metrics would you recommend that help me to see when the vibe turns in one of the communities?

*Found another metric that is very interesting and descriptive in regards of the health of a community: The age. How long much time are your active users already on it. Is it too low it means that the content/community doesn't really compel a user to stay, is it growing by the day you are not attracting new users/new users get scared away. Just wanted to add this.

  • 3
    Can you specify what kind of sub communities you have, please? Are they formed around a interest you have in common? Or is it a customer community where people seek help for products you sell? – Zerotime Aug 22 '15 at 21:01
  • It is about a shared interest but then the users organize themselves mostly due to the location they're at. So most of the sub communities are simply users with the same interest who want to talk with the people of the same area. (BTW sorry for the late answer) – loiro Sep 2 '15 at 9:08
  • 1
    Are we talking about online or physical communities? You mentioning physical factors as location and location preferences doesn't sum up with the CHI which is a "Community Health Index for Online Communities" I'm sorry if I misunderstood anything but your comment is quite confusing to me. – Zerotime Sep 2 '15 at 10:21
  • Hey Zerotime, no problem :) So they are online communities of interest. Just the location is used to create subcommunities so the content is more relevant and the user can meet up and stuff – loiro Sep 2 '15 at 12:18
12
+50

I'm pretty sure we will never know of a satisfying answer as this is a question every community manager or leader is concerned about. I'm also pretty sure that my answer won't satisfy you.

Let's start with something really basic, at least something basic to me. I think it's highly unethical to judge your community by using numbers. Numbers are related to economics, and if you rate or measure the potential of your community, you just want to know if it can compete against something or if it's capable of buying stuff. De facto, there's no universal scale which will tell you when a community is good or bad and there will never be because humans are unpredictable.

Let's talk about the Community Health Index of Lithium. The first thing you see in the table of contents is the "Executive Summary".

In the current economic climate, companies are discovering that their online communities have become a powerful and cost-effective vehicle for interacting with customers. For example, a consumer electronics community that runs on the Lithium platform recently reported 1.4 million deflected support calls, resulting in an annual estimated savings of $10 million.

From this point on, I can pretty sure say that the Index is a business approach. You have a community? Fine, Lithium is gonna tell you how to earn money with it or how to save money with it. It makes sense to measure a community in numbers to analyze its economic potency, to figure out if it can compete with other communities in the same field. The Index will definitely help you to find out what economical direction your community is going but it won't tell you where it's going emotionally and socially since this isn't a big enough value in the newly discovered term of Online Communities.

If you discover something new, corporates will try to figure out how to make money out of it. It's normal to see something like this, and they find a way to do it eventually, nevertheless they will ignore other facts for the sake of money. I will explain this at the "measurements" of Lithium.

  • Members
    • Few or many members say nothing about problems. It means that you're community is still growing or that the interest your community is built on generally has a small or large audience. You can't calculate anything with the member value as it says nothing about the feelings of the community.
  • Traffic
    • It's typical. Low traffic, less opportunities to promote stuff, higher traffic, more opportunities to promote stuff. Traffic is a number which also doesn't help to find problems and possible solutions.
  • Content utility (a function of the number of posts and the views)
    • "How many views did you get on your last article?! Nevermind my question, it's too low, anyway as you can always increase it!" An open-ended way of measurement.
  • Responsiveness (how fast does a post get interactions)
    • Finally something interesting as it shows if users appreciate your community that much that they try to keep up to date all the time.
  • Interactions (unique visitor participators and comments)
    • "How many interactions did you get on your last article?! Nevermind my question, it's too low, anyway as you can always increase it!" An open-ended way of measurement.
  • Liveliness (how much activity)
    • Quite broad. Perhaps, it can help to find some spots.

We see that at least four of the six ways of measurement of the Community Health Index are thought to measure your economical strength. Not many members? Lithium isn't interested. Not enough traffic? Lithium isn't interested. Not enough content utility? Lithium isn't interested. Not many interactions? Lithium isn't interested.

There's not this way of measuring your community socially, economically, culturally and emotionally. If there was, politicians jobs would just have gone. These people try to manage the largest communities of the worlds, also known as nations. I'm not a politician but I'm sure that such an Index might not work.

Summarizing this all, everything I can tell you out of an ethical point of view is to get into contact with your community as often as possible and as personally as possible. If you want to know if your community is struggling with anything, just ask them, they will complain earlier or later.


Let's get to the more economical approach of this answer: I'll try to modify the Index a little bit and add some values and ways of "measurement", however, please don't forget that I'm not paid to do so and that the development of the CHI took some time.

Everything of the current CHI plus some own points:

  • Members
  • Traffic
  • Content utility
  • Responsiveness
  • Interactions
  • Liveliness
  • Ratio of posts and reports
    • Obviously, it's important to keep this tracked. If it rises, your rules are too loose or may be easy to bypass. It creates unrest in your communities as low quality content keeps popping out. This will hurt your growth a lot!
  • Ratio of posts to members
    • Do you attract a lot of new users or old, and known ones? If it's the latter, the entry barrier might be too high as the same users keep posting and new users just don't get the hang of something specific.
  • The time it takes to register / sign up
    • The more it rises, the more hassle it is to sign up. Do you want to spend 10 minutes just to get access to this one post?!
  • The availability of staff
    • Staff has to be reachable and available at almost every time. The moment people aren't able to contact you, they will lose interest and think you don't care enough. Instant gratification is important in this time and age. Generation Y doesn't like to wait for you, they will just leave then.
  • The length of posts
    • Longer posts indicate more engagement, shorter posts indicate casual conversations. This helps you to determine if your community is fast-paced or slow-paced. Slow-paced communities usually are built up on the interests of politics, science and everything that needs some time to understand it. Fast-paced communities usually are built upon celebrities, games or movies.

I added these four because I think that they are a stable way to "measure problems". Of course, I couldn't conduct any research and these points may turn out false and untrue. You should take into account Generation Y. They are valuable customers, they will buy almost everything if you can deliver on time. A lot of things are about time, and you know the proverb "time is money". Every second you lose in cluttered UX or in a pointless discussion with staff is lost investition time.


However, I want to remind you that I basically think the same as @AnnonomusPenguin: There's no automated way to measure your community. While I don't think it's random how "high the quality" of your community is, I completely agree. If you really want to know the health of your community, ask them. They will tell your better than any number could ever do. What do politicians? They hold polls to get the opinion of the people, they ask them what they want to see changed. A number can tell you a lot, a member can tell you even more.

  • 2
    Hey Zerotime, first of all thank you for the great answer. Since I posted the question I started to implement s Reports/Posts ratio and started analyzing the post per member ratio which is not that hard to do but helps unbelievably well. Just wanted to add that even though I do want my communities to work economically (server costs etc.) as well, this isn't my main focus. So in the end everything I read and also your answer really got me to the point where I think the only stable way to know what is going on is to elect Mods and communicate with them often and openly. – loiro Oct 1 '15 at 11:11
2

I don't know if this is helpful as it may require you to get more software but thinking about this might.

https://en.wikipedia.org/wiki/Centrality

There are various measures of centrality for nodes (people) in your network. Some of them are critical as they are a hub of a social network (think cliques). If one of those popular people leaves then a disproportionate amount of people will follow.

It's important to know when a popular person is waning as that may signal an exodus.

Also a more robust community will be more resilient to folks leaving.

The idea is to count the number of hubs in your networks by the various centrality measures and track them over time.

  • Thanks a lot for the answer. Yes I thought about that as well but as you said it will be a major change to implement it. That's why I thought about an easier thing like measuring the amount of posts, flagged posts, engagement etc. But you are right, we should at some point include a system oriented at graph theory. Do you use something like that and have experience in it? – loiro Sep 7 '15 at 12:06

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.