Leaderboard Prejudice

Sanya Weathers, an blogger of exponential blogs, has started writing over at the community manager watering hole, Metaverse Mod Squad.  In the latest post she discusses the fiscal responsibility of the community management team (not without comparison to another department), but the crux of the post is on Leaderboards.  Sanya writes that “achievement, respect, and a sense that time spent on your product is not time wasted can all be checked off with leaderboards, or whatever you want to call your comparative ranking system.”  Dan Gray at his blog echoes Sanya’s thoughts by saying “it’s free content, it’s recognition, and it’s a fantastic tool for your community.”

There is a dark side though.  A darkness that harms community.  We shall name it “elitism.”

Leaderboards are very fun.  Take, for instance, Turbine’s leaderboards for their relatively new skirmish system in Lord of the Rings Online.   The leaderboard captures most points, most points per hour, success rate, and so on.  The leaderboard can be filtered by skirmish type, classes, level range, and even kinship (guild).  A lot more data is actually published than ranked, and in-game players can see a ton more statistics.  Following Sanya’s suggestion, there is no reason not to also rank things like number of Lieutenants killed.

Another fun one in Guild Wars was published a long time ago, with only the top ten, was the people with the most Sunspear and Lightbringer points.  Now this was in the waning days of The Scribe articles ArenaNet was posting, and in hindsight it appears they were scrambling for content.  But, I agree with Sanya that it was fun to see nonetheless (even if I wasn’t listed as person 483,237).  What both showings have in common is that they are fun statistics with little showing of worth as a player.  For example, the person with the most Sunspear points (133,166 M B End) had nearly three times the amount necessary to gain all mechanical benefits.

The harm to community arises when the community starts to believe that somehow the leaderboards show a person’s inaccurate worth as a player.  Take Guild Wars Hero title, which is a tiered ranking system showing the amount of Fame a player has received from playing Heroes’ Ascent.  Really what it shows is how much time that player has spent in Heroes’ Ascent.  Players forming teams for PvP began to exclude anybody below a certain rank because it was an easy way to find “better” players.  Players with hours worth of Fame were simply snubbed because of the ranking system.

There is also the Gear Score in World of Warcraft, which calculates arbitrary numbers to show a player’s equipment.  The community took it upon themselves to use this tool to rank each other.  While on some level the Gear Score can actually help determine that a player’s equipment simply is not good enough for the specific challenge, there have been plenty of posts on the elitism the ranking system created.  All Blizzard can really do is laugh at themselves.

I think that solution is partly in Sanya’s list of helpful tips in that developers should rank everything.  The more information players have the less chance that the community will hone in on one number.  I would love to see the amount of kobolds I’ve killed in Dungeons and Dragons Online, or the amount of times I have used Rallying Cry in Lord of the Rings Online, or the number of feathers I’ve looted in Guild Wars.  The amount of information would just crush the creation of a simple score that defines the worth of player.

–Ravious
you judge too easily, Merlin

14 thoughts on “Leaderboard Prejudice”

  1. Leaderboards don’t lead to evaluation-mania, failure does. Failing stinks, and if you have groupmates that just weren’t up to a raid or dungeon there’s little you can do to bring success. That’s why players feel the need to evaluate others, and of course it then occasionally gets out of hand.

    I wrote an entire post on this a while ago, but the title – “Why can’t Blizzard be the bad guy?” was really my entire point. Players make bad decisions we’re not only forced to make the decisions ourselves, but make the decisions with imperfect data. But it should be the game excluding people, not other players.

    For example, you should literally not be allowed to DPS a heroic dungeon in WoW until the game measures you doing at least 1600 DPS on a target dummy. Once you’ve done it then the game considers you ‘attuned’ and you can physically enter the dungeon. There could be similar tests for tanks and healers.

    Not perfect measures, of course, but it would drastically curtail the kind of “Lord of the Flies” thing that players have going these days.

    That or make the game a little more failure-friendly, but that’s another long post.

  2. Boatorious, I actually described a system similar to what you’re describing in a recent post. However instead of gated attunement, the result of these “Role Proficiency Tests” is in fact a rating, which would in fact be the new gearscore and lead to elitism. The difference is that this elitist statistic would be based on a standardized, skill based gameplay test of a character’s proficiency in their declared group role. The tests would double as tutorials for those who can’t seem to raise their rating, leading to the education and general improvement in the playerbase and PUGs.

    Elitism has a place in cultures as long as its net effect is driving people to learn and better themselves, instead of giving up because they came late to the party.

  3. I always loved the Camelot Herald and the rankings it listed. I used to have a screenshot of me being on one of the top RPs for the week (for a certain class too, but, cmon, I WAS ON IT STILL!). Good times. I wish WoW had more in that area, here’s hoping to rated BGs in cataclysm.

  4. I am a pretty good healer, but WoW was letting me into the ICC 5 mans and even heroics through the random dungeon finder when I was in mostly greens. We would fail dismally and I had no idea what was going on until someone checked my gear. The minimum checks they have in place are not enough, while gearscore elitism is of course too far in the other direction.

  5. One of the things that I experience with gear score is that the standards are far higher for tanks and healers than for DPS. The quality of your tank and your healer are far more important than the quality of your DPS in most WoW 5-man instance content. So you end up with a situation where some horrifically geared rouge (misspelling intentional) is whining about how the tank or the healer isn’t geared enough and wasting his precious, precious time. Time he could be using to… get carried through some other heroic in order to maybe someday figure out that hit is more important to rogues than crit?

  6. When I first encountered Leaderboards back in DAOC I found them moderately amusing. I still think they are worth a glance now and again, especially on long griffin rides and the like.

    In general, though, there’s very limited pleasure to be gained, for me at least, in looking at a lot of recorded data. Much more fun to write an in-character diary and take screen-shots. More work, but a lot more satisfying and evocative years down the line.

  7. Leaderboards have their place, and if implemented correctly they can be a tool of recognition and retention. If used incorrectly they can become tools that create barriers and frustration for players. Guild and player rankings, stats, and achievements I think are positive things to have. Gear Scores though I think toss up barriers to entry because player X will simply be excluded if his gear score isn’t optimal.

  8. Definately like your idea of ‘feathers looted’. Gear score is too general and elitist. Accomplishments within a healing class is totally different then that of a tank class and need to both be in their own separate lists. As well as general accomplishments that any class can achieve. Still like in DAOC guilds and individuals and individuals within a class need to be recognized for pvp accomplishments.

  9. Regarding HA, specifically:

    Had the Hero title track had been removed HA would have become a ghost town pretty quickly, with players moving to the more rewarding and mechanically sound Guild Battles. There was certainly an issue of rank elitism that created a barrier for new players, but if you remove the rank there’s very little reason to play there.

    The problem is a poorly implemented ranking system. In this case, the grind element that excludes new players simply on the merit of them being new.

    There’s a pretty elegant way to counter that which I’ve not seen used outside of Fury: We implemented both quantitative and qualitative ladders. Quantitative ladders were your usual fare, while qualitative ladders used only the average of your top 10 results, neutralizing the grind element that many ladders are plagued with. (Even competitive ladder systems like ELO have a bias toward sheer volume over quality.)

    In the case of HA that would mean looking at the 10 runs in which you earned the most fame, and averaging that out to your total score.

    I’d love for a similar system in Guild Wars 2. Here’s hoping.

  10. Weekly accomplishments are nice as well. Who earned the most ‘x’ separated into class/guild/overall rankings.

Comments are closed.