karma, n. the cosmic principle according to which each person is rewarded or punished in one incarnation according to that person’s deeds in the previous incarnation.¹

Though not always called “karma” (Everything2 has a multi-faceted system involving “reps” and “C!s,” and Slashdot calls it a “score”), the concepts all serve the same purpose: karma has come to be a way of quantifying the contributions of a person to a particular website.

In theory, your karma grows as you make insightful, interesting, or useful comments – anything that contributes to a conversation in a constructive, even if disagreed with. Ideally, that is a great idea, and ideally, any mis-scored commentary would be self-correcting thanks to the generally large number of people reading these comments.

Sadly, in real life, this doesn’t pan out. People, no matter whom, no matter what community, are susceptible to group think. Even in places such as Hacker News, where the average user seems to be fairly well-spoken, group think can – and does – occur.

Worse yet, there is a much more serious issue: that of everyone using karma appropriately. It seems to be that some people use karma as a “like” or “dislike” system. If someone disagrees with a comment, often times they give it negative karma (a.k.a. “vote it down,” or whatever else you would like to call it). There has been much discussion on this topic on social websites such as Hacker News or Reddit:

• I believe Karma should be awarded for posts that generate comments. Conversation starters should be as valuable to our community as a funny pic that’s upmoded.
• Dear HN, I’m worried about us

All this discussion is what spurred me to write this article in the first place. I’ve seen much debate, and even some attempts at improvements:

What I’m surprised to have not yet seen is what I am about to suggest: stop quantifying karma!

I know how great it is to have a “karma score” and watch it go up (or down), but people view those little arrows with so many varied meanings that often, they’re as good as useless. Why not simply provide a system of karma that qualifies karma? Give user’s and even comment’s karma an evaluative rating (perhaps, relative to everyone else): very poor, poor, neutral, decent, good, very good, excellent.

What good does this do, you say? I think (though I have not done any formal studies) that it could potentially eliminate at least part of the issue: the ambiguity. The karma ratings now have meaning to them, and people will think twice before rating a comment (and consequently a user) “poor.”

Now, that isn’t to say that on the backend, there couldn’t be a numeric score. Sure, implement it with a quantitative, integer-based system on the backend, but just don’t expose it to the user. It turns karma into too much of a game, and less of a way to determine the quality of a user’s contributions.

The closest system I’ve seen to this is Slashdot’s system:

While the scores do not accumulate (which does seem advantageous in its own right), the scores themselves are categorized. They are on a limited scale from 1-5 (so even if you make the wittiest comment, you can still only score a maximum of “5, Funny”), and scores are qualified.

As well, karma on Slashdot is a limited resource. Users periodically receive a limited number of points they can use to rate comments. What does this mean? It means that users are much more careful about using their points. Sure, some users use them spitefully and mis-rate comments, but since the scale is limited to only 5 numbers, it is possible for it to self-correct. With an infinite karma system (meaning your score can keep going in either direction), there is a tipping point where a comment is unlikely to recover from false ratings. Consequently, since points are limited, it becomes less likely that you will have many users wasting their points rating down a comment spitefully (a reduction in group think).

To point out a particularly ineffective system, there is Digg. Here, karma is only localized to the various posts – much like Slashdot, but with a very detrimental exception: no qualification to the ratings.

Sadly, this seems to be the worst of all worlds. Since the comment karma is on an infinite scale, and it isn’t cumulative for a user in any way, users have no reason to think twice about posting a comment. Posted a troll-esque comment and got a bad score? No big deal, since it doesn’t reflect on you. From the other side: don’t agree with a comment? Just vote it down! I believe this is partially the reason that the discussion quality on digg is quite low – no checks are in place, and the site has long since reached critical mass.

So this is my argument to all of those social websites out there using comment and user karma systems: don’t let it be a game. Qualify, don’t quantify. You might even go so far as to make comment karma a limited resource. It might just work out for the better.

¹ Dictionary.com Unabridged (v 1.1).

Posted on Thursday, February 5th, 2009 at 8:32 pm, under thoughts.

Follow reponses with the RSS 2.0 feed. Both comments and pings are currently closed.