karma, n. the cos­mic prin­ci­ple accord­ing to which each per­son is rewarded or pun­ished in one incar­na­tion accord­ing to that person’s deeds in the pre­vi­ous incar­na­tion.¹

Though not always called “karma” (Everything2 has a multi-faceted sys­tem involv­ing “reps” and “C!s,” and Slash­dot calls it a “score”), the con­cepts all serve the same pur­pose: karma has come to be a way of quan­ti­fy­ing the con­tri­bu­tions of a per­son to a par­tic­u­lar website.

In the­ory, your karma grows as you make insight­ful, inter­est­ing, or use­ful com­ments – any­thing that con­tributes to a con­ver­sa­tion in a con­struc­tive, even if dis­agreed with. Ide­ally, that is a great idea, and ide­ally, any mis-scored com­men­tary would be self-correcting thanks to the gen­er­ally large num­ber of peo­ple read­ing these comments.

Sadly, in real life, this doesn’t pan out. Peo­ple, no mat­ter whom, no mat­ter what com­mu­nity, are sus­cep­ti­ble to group think. Even in places such as Hacker News, where the aver­age user seems to be fairly well-spoken, group think can – and does – occur.

Worse yet, there is a much more seri­ous issue: that of every­one using karma appro­pri­ately. It seems to be that some peo­ple use karma as a “like” or “dis­like” sys­tem. If some­one dis­agrees with a com­ment, often times they give it neg­a­tive karma (a.k.a. “vote it down,” or what­ever else you would like to call it). There has been much dis­cus­sion on this topic on social web­sites such as Hacker News or Red­dit:

• I believe Karma should be awarded for posts that gen­er­ate com­ments. Con­ver­sa­tion starters should be as valu­able to our com­mu­nity as a funny pic that’s upmoded.
• Dear HN, I’m wor­ried about us

All this dis­cus­sion is what spurred me to write this arti­cle in the first place. I’ve seen much debate, and even some attempts at improvements:

What I’m sur­prised to have not yet seen is what I am about to sug­gest: stop quan­ti­fy­ing karma!

I know how great it is to have a “karma score” and watch it go up (or down), but peo­ple view those lit­tle arrows with so many var­ied mean­ings that often, they’re as good as use­less. Why not sim­ply pro­vide a sys­tem of karma that qual­i­fies karma? Give user’s and even comment’s karma an eval­u­a­tive rat­ing (per­haps, rel­a­tive to every­one else): very poor, poor, neu­tral, decent, good, very good, excellent.

What good does this do, you say? I think (though I have not done any for­mal stud­ies) that it could poten­tially elim­i­nate at least part of the issue: the ambi­gu­ity. The karma rat­ings now have mean­ing to them, and peo­ple will think twice before rat­ing a com­ment (and con­se­quently a user) “poor.”

Now, that isn’t to say that on the back­end, there couldn’t be a numeric score. Sure, imple­ment it with a quan­ti­ta­tive, integer-based sys­tem on the back­end, but just don’t expose it to the user. It turns karma into too much of a game, and less of a way to deter­mine the qual­ity of a user’s contributions.

The clos­est sys­tem I’ve seen to this is Slashdot’s system:

While the scores do not accu­mu­late (which does seem advan­ta­geous in its own right), the scores them­selves are cat­e­go­rized. They are on a lim­ited scale from 1–5 (so even if you make the wit­ti­est com­ment, you can still only score a max­i­mum of “5, Funny”), and scores are qual­i­fied.

As well, karma on Slash­dot is a lim­ited resource. Users peri­od­i­cally receive a lim­ited num­ber of points they can use to rate com­ments. What does this mean? It means that users are much more care­ful about using their points. Sure, some users use them spite­fully and mis-rate com­ments, but since the scale is lim­ited to only 5 num­bers, it is pos­si­ble for it to self-correct. With an infi­nite karma sys­tem (mean­ing your score can keep going in either direc­tion), there is a tip­ping point where a com­ment is unlikely to recover from false rat­ings. Con­se­quently, since points are lim­ited, it becomes less likely that you will have many users wast­ing their points rat­ing down a com­ment spite­fully (a reduc­tion in group think).

To point out a par­tic­u­larly inef­fec­tive sys­tem, there is Digg. Here, karma is only local­ized to the var­i­ous posts – much like Slash­dot, but with a very detri­men­tal excep­tion: no qual­i­fi­ca­tion to the ratings.

Sadly, this seems to be the worst of all worlds. Since the com­ment karma is on an infi­nite scale, and it isn’t cumu­la­tive for a user in any way, users have no rea­son to think twice about post­ing a com­ment. Posted a troll-esque com­ment and got a bad score? No big deal, since it doesn’t reflect on you. From the other side: don’t agree with a com­ment? Just vote it down! I believe this is par­tially the rea­son that the dis­cus­sion qual­ity on digg is quite low – no checks are in place, and the site has long since reached crit­i­cal mass.

So this is my argu­ment to all of those social web­sites out there using com­ment and user karma sys­tems: don’t let it be a game. Qual­ify, don’t quan­tify. You might even go so far as to make com­ment karma a lim­ited resource. It might just work out for the better.

¹ Dictionary.com Unabridged (v 1.1).

Posted on Thursday, February 5th, 2009 at 8:32 pm, under thoughts.

Follow reponses with the RSS 2.0 feed. Responses are currently closed, but you can trackback from your own site.