Introducing karma
Posted by Adrian Holovaty
August 20, 2004 at 12:19 p.m.
We just launched a new comment-related feature on lawrence.com -- user karma. It's a simple way for the community to monitor itself by promoting insightful comments and demoting the, well, not-so-insightful ones.It works like this. If you're [logged in][1] to your lawrence.com account, you should see a "Was this comment interesting?" note, with Yes and No links, below each comment throughout the site (e.g. blog comments). Click on the Yes or No to instantly grade that comment. It's kind of like [Amazon.com][2]'s user reviews.The site keeps track of how many people voted Yes vs. No. for each comment and displays that info above each comment. Also, each lawrence.com user that has been voted-on at least three times will have a karma bar next to his or her username. That bar represents the user's overall karma -- how often other users have found that person's comments to be interesting. The fuller the bar, the better the karma.The exact formula for how we calculate a person's karma is secret, but it has to do with how other people have rated that person's comments over time, along with a number of other things. We'll be tweaking the formula as time goes on.Note that you can't give yourself karma (for obvious reasons). Also, as we mentioned above, you have to be logged in to give people karma. Without that login requirement, everyone would give everybody else, and the karma system would be worthless.What's the point of karma? It's to help people sort through comments and get to the more highly-ranked ones more quickly. Hopefully it'll encourage more thought-provoking commentary, discourage trolling/flaming and result in an overall higher-quality discussion on lawrence.com. It'll be cool to watch what happens.UPDATE, Monday, Aug. 23: Due to overwhelming feedback, we've removed karma from lawrence.com blogs and kept it on lawrence.com reviews. See [this comment][3] for more. [1]: http://www2.lawrence.com/users/login/ [2]: http://www.amazon.com/ [3]: /behindthescenes/2004/aug/20/karma/#c7322
Comments
Shelby 18 years, 7 months ago
I like it I guess, but I also dislike it. I don't know why.
This comment sucked.
adrian 18 years, 7 months ago
Thanks for all the great input, guys. We've taken your comments to heart and removed the karma stuff from lawrence.com blogs.
We're still keeping the "Was this a good comment?" functionality on other comment-enabled parts of the site, such as movie reviews, where the comments are less of a conversation and moreso standalone pieces. In those cases, we'll soon begin to offer a way of displaying the reviews in order by their rank (e.g. "Sort these reviews by most popular"), which we think would be a useful feature to have.
Patrick Quinn 18 years, 7 months ago
Man O' Leisure: Ditto.
Jason Barr 18 years, 7 months ago
Liz and I just commented at the same time and I can only vote if her comment is interesting. Does this mean that only the most recent comment can be voted on. That seems a little odd. -jbarr I voted for you liz, I hope that makes you feel better. I see your point too.
liz 18 years, 7 months ago
I like the spirit of the idea. However, I'm just worried that it will make people less likely to comment at all for fear that people will mark them as "uninteresting." We'll see.
Rob Gillaspie 18 years, 7 months ago
Good point, Leslie.
It doesn't seem to promote discussion so much as perpetuate the whole idea that these comments sections are a popularity contest. Which they are, really, but why rub it in?
PayTime 18 years, 7 months ago
If nothing else, it just gives another number to how many people may viewing the blogs at any given time. I would like an over all counter for each page download of the individual blogs. I heard a rumor that was coming down the road. Is this true?
Signed Payton Burdough
MAMAT 18 years, 7 months ago
What does one do about no karma? LOL
liz 18 years, 7 months ago
So what will staying out of the voting completely do for someone's karma? I just can't bring myself to participate for some reason. I may be mixing my eastern and western religious philosophies, but I feel like all this "judgement" is bad for the soul and couldn't possibly be a good karma building strategy. I also think it distracts from the discussion. Could this concept be done with something besides a comment by comment tally?
simon 18 years, 7 months ago
quisp: I promise you that the rules for Karma are the same for us as they are for everyone else. It's against our interests to mess around with the system - it would be unethical, and it would also be extremely embaressing if we were found out.
Tim vonHolten 18 years, 7 months ago
i think it will be helpful to finally know what is interesting.
Jason Barr 18 years, 7 months ago
Hmmmmmm, will there still be deviders between the posts? It seems a little hard to read without something seperating the posts. However, I do like the idea. -jbarr
lazz 18 years, 7 months ago
john ashcroft's idea? see ya, kids. it's been fun.
barr 18 years, 7 months ago
This idea also seems like it could just turn into something really partisan. I can just see people voting no on people they don't agree with. It also seems a little distracting. This is it. I'm going to stop commenting. bye. - Team Wolf
adrian 18 years, 7 months ago
1981: It's not possible to vote for yourself, which is why you didn't see the voting thing for your own.
quisp 18 years, 7 months ago
I noticed that Simon automatically started out with a full bar of Karma. Interesting...
leslie 18 years, 7 months ago
I'm already eating my words. I voted because I was annoyed at how low David Ryan's karma was. Guess I'm exacerbating the cliquishness dilemma. Can I openly campaign to have my karma raised?
wilson 18 years, 7 months ago
I think it's worth reiterating that this whole karma thing can fairly be considered an experiment, and really doesn't have to affect how you use the site at all. I think it will be interesting to see if it proves useful, and it's pretty easy to ignore in the meantime.
leslie 18 years, 7 months ago
I appreciate the sentiment in this, and I know you guys are working hard, but I don't see myself voting for or against someone's comment because I assume the data and my identity are stored somewhere in the system...which kind of creeps me out. Frankly, I've found it rather easy to determine who is an asshole and who isn't via their comments alone.
When we weed out the meanies, can we have a sex blog again?
adrian 18 years, 7 months ago
Good point about the comments being a bit harder to read. We'd love to hear more feedback about how it could be improved. One thing we've been considering is putting all the karma stuff in a left column, bulletin-board style, and putting the comments themselves in a right column.
adrian 18 years, 7 months ago
Yeah, combatting the cliquishness is key...We're going to roll in some safeguards in karma calculation that will attempt to avoid that, but we can't explain the techniques too much or they'll be abused. :-)
Like Simon said above, this is all still in the experimental stage, so we'll constantly be tweaking it.
edie_ 18 years, 7 months ago
I can appreciate the "X of Y people found this comment interesting" feature but could you explain why the "karma" calculation system is secret? If no one knows what the criteria is for it, how does it hold any meaning? I'd like to understand more about that because right now it seems like it bears the threat for the Journal-World to discredit controversial writers.
simon 18 years, 7 months ago
leslie: that's a very valid point, and in fact you're right: the system does keep track of who voted for what, in part so that we can ensure people don't vote for something more than once but also because in the long run we want to incorporate this data in to the karma calculation - for example, if someone ALWAYS votes negatively we'll count their contributions less than someone who votes negatively half the time and postiviely the other half.
So when you're voting, please bare in mind that the votes are only semi-anonymous. Sorry for any confusion - we should have made this clear from the start.
simon 18 years, 7 months ago
Staying out of voting entirely means you won't have any affect on other people's Karma.
liz: we've been talking about this and we've decided that Karma may not be a particularly good term for this whole thing either. Philosophically, Karma is something that you accumulate through your actions, not through people judging you. We're going to leave the system as-is over the weekend to let people play around with it and to get an idea for how well it's working, but we'll almost certainly make significant changes (including a possible renaming) early next week.
simon 18 years, 7 months ago
We're keeping the karma formula secret for the moment for two reasons: firstly, to discourage people from "gaming" the system (trying to trick it in to giving them the highest possible karma) and secondly because we will definitely be changing the formula several times in the next few days to discover what works best. Once we have a better understanding of how the karma system is working we may decide to share how the calculations are made.
Our end goal with the karma system is not to discredit people or set them up against each other; it's to encourage people to contribute to the community in a positive way. It's not an easy problem, so the more help we get in terms of feedback and suggestions the better!
lazz 18 years, 7 months ago
I apologize for making a rash comment and blowing off the concept without watching it in action. I'm cranky today and this idea just struck me wrong. It does seem to me that it will re-enforce some cliquishness that already exists here, and I think that each reader should be given an opportunity to independently judge the value of the posts; I also think that some of the worst posts can lead to some of the best discussions -- we need to see some of the kooky, irrelevant stuff, because it informs where some people are coming from, and then we get a chance to react to it. But again, I barked without giving it fair consideration, I'm wrong for that, and I apologize. I'll be eager, along with everyone else, to see how this might affect our lively little online conversations.
ManOLeisure 18 years, 7 months ago
I don't post a ton, but as a teacher and a writer, I still don't quite understand how you wordsmiths decided on the term "Karma."
Frankly, the way in which you are using the word isn't really even close to karma's most informal definition.
I understand the need to hip and with-it, but forcing thoroughly-misrepresented Eastern-spiritual cool into your site by providing a visual judgment of a comment before one reads a post insults the intelligence of your readers.
In addition, could you perhaps come up with a more milquetoast phrase than "good comment?" My suggestion for your KARMA after seeing it in action - drop it. There are enough bells and whistles here - radio, trailers, rollicking, incisive blogs, funny stuff - so let the forums alone. Remember: if you ever have to compare something you've come up with to something Amazon does (and in the second graf of your Karma Manifesto, no less), chances are your case hasn't been sufficiently thought through.
leslie 18 years, 7 months ago
I'm wondering if eliminating the "0 of 2 people thought..." line would be an attainable middle ground? I'm personally concerned about the Eastside book club, which will start posting in a couple of weeks. "Good" comments or not, I want everyone who read the book to be comfortable posting. I'm afraid that if someone gets a zero on their comment, they will never post again on future books. Which is mighty bad for dialogue. An unpopular comment is not necessarily bad for the discussion.
quisp 18 years, 7 months ago
It's a confusing and needless addition, IMO. It will create an "us and them" mentality. Wholesome and safe comments will be treated like golden children, and the rest of you will be banished to the corner. Just watch. I was just thinking this site wasn't factional enough. Thank you for solving that problem for me.
liz 18 years, 7 months ago
I'm not sure that "interesting" is the best term to base this system on. As monkeywrench's ironic comment pointed out, what people find interesting is pretty subjective.
adrian 18 years, 7 months ago
We've changed it to the deliberately-vague "Good comment?" Thoughts?
simon 18 years, 7 months ago
Please bare in mind that this is an experimental feature - although we're keen on having some kind of karma system, we are very aware that it could have both positive and negative effects on the community spirit. We will be tweaking the system continuously to try and find the best possible solution.
We actually discussed a number of ideas for minimising the chances of people "gaming" the system - voting against people they don't like or disagree with - but in the end we agreed that it would be much easier to fix any problems once we had some real live data to look at. The system as it stands now is a first draft, and will certainly change as we gain a better understanding of how people are using it.
Keep the feedback coming, but please don't write the system off without giving it a fair chance. Constructive criticism is welcome - that's the main reason we started the Behind The Scenes blog.
lori 18 years, 7 months ago
I second Leslie's comments; I can pretty much tell on my own if something is worthy of the "good comment" rating, without knowing that 4 of 5 agree with me. I'm not sure how helpful they could be. I don't know; it seems like nasty flashbacks of high school. People voting on whether they like what you say or not; their judgement deciding your karma. So far I can't say that I think these changes are improvements. But, in the grand scheme of things, it's not that big of a deal one way or another, and maybe after some kinks are worked out, these changes will add another bit of character and interest to the site. I'm just getting older, and I guess my geriatric mind just doesn't handle change that well anymore.
And as 1981 points out, it is a bit more difficult to read. It seems very busy, if that makes sense; and for me, anyhow, the comments seem to run together. If dividers could be used, that might make the comments easier to distinguish from one another.
In any case, no matter my opinion on the changes, I do appreciate the site and all the effort y'all put into it.
edie_ 18 years, 7 months ago
quisp: I think I heard there was a conference at the J-W last week. If you've ever been to an event like that you know that besides muffins and coffee these toe tapping hoedowns are the catalysts for new and exciting changes.
Also when noticing the full erections of karma behind the J-W's you may recall they invented the rules of the karma system. The readers would abuse the rules if they knew them. This summary of their explanation of the rules should be enough for you: Karma...GOOD...low karma...BAAAD.
Any questions? I'm thinking this particular comment will lower my karma.
I'm a bright gal but I'm not really into being a teacher's pet. This full bar is cramping my style.
edie_ 18 years, 7 months ago
Phew! Now my karma's down I can respect myself again.
To support Leslie's comment and add my own... I have times when I feel annoyed over commentators who blurt things out stupidly or who express a different belief system than my own. But I'm GLAD they can air their opinions or experience. I want to hear what people have to say.
If the J-W really wants to take the higher ground on this they'll come out and defend free commenting and expect some irrelevant remarks to be the trade off for open discussion. Readers can be respected to make their own decisions about comments without a rating system. Besides what's to hold your interest if it were a site exclusively full of middle class college suburbanite dialogue anyway? It should be to the site's credit that it's NOT a bunch of whitewashed blither blather...unless the goal is to steer the site into that direction. I'd be surprised if any of the individual journalists supported that. I don't know about the J-W ownership or upper management though.
Commenting has been disabled for this item.