Randy Farmer and Bryce Glass write about designing online reputation systems, karma, and points, tied to their O'Reilly book.
HN commenters recommend the site and book as the go-to resource on reputation systems, praising its ideas on how points shape user behavior.
Here's a story about The Sims Online I posted a few years ago, in which I described making a maze solving bot to quickly and automatically generate millions of Simoleons, and an ad-hoc solution to the delivery problem:
https://news.ycombinator.com/item?id=11730181
I wrote a bot for The Sims Online that manufactured Simoleons, then repurposed it to catalog and describe objects in the game for you with a speech synthesizer.
TSO had some ham-fisted money making multi player games that forced players to interact with each other in exchange for Simoleons.
One of them was a maze solving game for two players, where one player is lost in the maze and can see the local walls around them, and the other player can see an overall map of the maze without the player's position, and has to guide them around and figure out where they are and how to guide them out by asking them what they see and telling them which direction to move.
That required a lot of work for two people to do manually, so there was a big reward, but it was trivial and fun to solve mazes programmatically! (Plus it made cool bleeping and kaching sounds as it solved the mazes and printed money!)
I would run two TSO clients at the same time, logged into different accounts in different windows. The bot attached to both of them, then screen scraped pixels and injected events to repeatedly solve mazes by moving the player around until it identified where they were, solving for the shortest path, and bringing them…
First off, for anyone looking at web reputation systems, read the book on the subject: Randy Farmer and Bryce Glass, Building Web Reputation Systems:
Book: http://shop.oreilly.com/product/9780596159801.do
Wiki: http://buildingreputation.com/doku.php
Blog: http://buildingreputation.com/
I can pretty much guarantee there are elements of this you're not considering which are addressed there (though there are also elements which Farmer and Glass don't hit either). But it's an excellent foundation.
Second: If you're going to have a quality classification system, you need to determine what you are ranking for. As the Cheshire Cat said, if you don't know where you're going, it doesn't much matter how you get there. Rating for popularity, sales revenue maximization, quality or truth, optimal experience, ideological purity, etc., are all different.
Beyond that I've compiled some thoughts of my own from 20+ years of using (and occasionally building) reputation systems myself:
"Content rating, moderation, and ranking systems: some non-brief thoughts"
http://redd.it/28jfk4
⚫ Long version: Moderation, Quality Assessment, & Reporting are Hard
⚫ Simple vote counts or sums are largely meaningless.
⚫ Indicating levels of agreement / disagreement can be useful.
⚫ Likert scale moderation can be useful.
⚫ There's a single-metric rating that combines many of these fairly well -- yes, Evan Miller's lower-bound Wilcox score.
⚫ Rating for "popularity" vs. "truth" is very, very…
All else being equal, people are more likely to follow people who already have lots of followers: there must be something good about their tweets that makes all those people choose to keep following them.
The number of followers is the only visible indicator of the "goodness" of an account. It's similar to karma on Hacker News, except that it's impossible to miss because it's prominently displayed. This is probably intentional on the part of Twitter, to grow their network.
http://buildingreputation.com/writings/2010/02/on_karma.html
Related:
Just yesterday I noticed that OReilly recently published Building Web Reputation Systems (Randy Farmer, Bryce Glass)
http://oreilly.com/catalog/9780596159801
http://buildingreputation.com/
Randy was consulting with a company I was at a couple of years ago and while the two of us certainly didn't see eye-to-eye on a lot of things, it encouraged a lot of thought and debate.
I'll be ordering the book today.
Original article this is pulled from (uncited in submitted post) http://buildingreputation.com/writings/2009/10/the_dollhouse_mafia_or_why_to.html
It's a little ridiculous that you can quote the meat of an article from another blog, write a short intro sentence, not cite the original and post it as insightful. I'll give him the benefit of the doubt and assume he just forgot to post the link to the original
There is a detailed description of the mechanism used in Yahoo! Answers in chapter 10 of the book. You can read the draft text here: http://buildingreputation.com/doku.php?id=chapter_10
BTW, everyone keeps assuming that reputation is all about karma. It's not. We cover that in chapter 1 (and over and over again in the rest...)
Cheers,
Randy Farmer, co-author Building Web Reputation Systems