« BayChi Talk Next Tuesday | Main | Dunbar Number Presentation at MeshForum 2006 »

August 11, 2006

TrackBack

TrackBack URL for this entry:
http://www.typepad.com/services/trackback/6a00d8341d8bc053ef00d8342a7f0d53ef

Listed below are links to weblogs that reference Using 5-Star Rating Systems:

» Christopher Allen on 5-Star Rating Systems from Geof F. Morris's Indiana Jones School of Management
Christopher Allen shot me an email the other day pointing to his work on practical applications of 5-star rating systems. I found it to be an interesting read, which you might expect given my previous commentary on the subject. Highlights: Thus even ... [Read More]

Comments

Feed You can follow this conversation by subscribing to the comment feed for this post.

F. Randall Farmer

See
http://web.archive.org/web/20010207224515/http://people.delphi.com/mike100000/p5summary.html#summary (or click my name below) for a summary of all the votes for the B5 Series.

Averaging all episodes you get a mean score of 8.19 with a standard deviation of 0.84 - a *very* narrow range.

Context and Motivation matter more than scale in getting reusable scores out of rating systems. As I like to tell the folks here at Yahoo! – the person creating the rating has to get something out of the transaction other than just altruism.

Adam MacDonald

I thought the issue, like Randy Farmer suggests, was that only positive discrete scales always tend to their maximum. The problem then in rating systems, like you mentioned I think earlier, is that readers have to interpret what the real norm and limits are. Even if every rating is in the upper quartile. You may want to check out what I did in Playerep (http://www.playerep.com), where the rating system is based on election but not a fixed scale (always norming to zero without influence). FWIW. Thanks.

Adam

Andy Tinkham

I like your classification scheme for iTunes. I'm going to have to work on adapting my scale (which is definitely showing signs of skewing towards the high end).

One thing, though. You mention that you can't create smart playlists to show unchecked songs. It's true that you can't do it with only one playlist, but you can do so with 2:

#1) Any playlist (for example, let's say your smart playlist for 4 star, checked songs, so you have it set (I assume) for My Rating = 4, Match Only Checked Songs = checked)

#2) Use these criteria: My Rating = 4, Playlist Is Not . Leave match only checked songs unchecked, and enable live updating and you have a smart playlist that shows you your unchecked 4-star songs.

Tino

>>Unfortunately, iTunes does not let you select only unchecked items, so I don't have a Smart Playlist for these; instead I keep them in a regular playlist.

You can:
Make a smart playlist called 'Checked' with a rule that is always true (e.g. artist does not contain (leave the field emtpy), or bitrate > 0). Create a smart playlist called 'Unchecked' with the rule: 'Playlist is not Checked'. There you have all your unchecked files.

From there you can make smart ones that use this Unchecked playlist.

I like your rating system. I have 40% on a 3star now, and around 25% for 2 and 4 stars. I might scale it down a little bit more.

hmm, after typing this I read the comments and see Adam McDonald already mentioned something similar about the unchecked items. I'll leave it here anyway...

Don Park

Christopher, I think some, if not all, of the issues with the 5-point rating system could be addressed with a better UI. For example, displaying a normal distribution curve skewed by accumulated rating to nudge raters away from the extremes. Of course, this doesn't work well if sample size is too small which usually happens on the far end of the long tail.

Jonathan

Thanks for writing this article. I found another view on 5 point music rating other than my own informational.

The comments to this entry are closed.

My Photo