Pareto Superior Testing

Our testers can veto releases at work, but we have an allied tradition that half a loaf is better than none. We may not get everything we want from an update, but if it makes some things better and no things worse, we go live. We can add the rest in a future update.

A gaming example comes from GW2 crafting. At launch, crafting could use items only from your character’s inventory. Soon after, you could craft from the vault but discovery was still inventory only. Now both check character inventory and the entire vault.

This is easier in my work than in gaming because our users are not competing with each other. If we can implement new functionality for one interface but need another month to accommodate the rest of our users, bonus for the users with the easy update. If your FPS added rocket launchers for PC players but needed another month to add it to the Mac client, forums would explode, especially if PC and Mac players were on the same servers. You can see this in games that are gradually rebalancing one class at a time rather than all at once. The relative values of classes are having large swings each month. LotRO had “the month of the [class],” TF2 had class-specific updates, and other games have similarly revamped single classes. See also City of Heroes gradually adding heroes’ passive archetype abilities over time, so there were months in which only half the classes had them.

Sometimes half a loaf is worse than none. Beyond the cases where it distorts your competitive balance, a function that only half-works can make some things worse and no things better. Adding something that only works for a known half of the users is inconsistent but reliable, which can be okay; adding something that works for everyone a seemingly random half of the time is inconsistent and unreliable, which is bad. The new functionality must work as expected, even if only under additional assumptions, and those assumptions must not cause other problems. Half a loaf is better than a whole loaf with gravel scattered through it.

: Zubon

Chris Carter, Joss Whedon, Google, and NCsoft

[Warning: there are some TV Tropes links in here.]

I have confessed to contributing to self-fulfilling prophecies: if you do not commit to something/one because s/he/it may not be around for long, s/he/it probably will not be around for long. So how do you invest yourself in something when the producers have a left a wake of unfinished and canceled projects?

Continue reading Chris Carter, Joss Whedon, Google, and NCsoft

Loot Bonuses: Bad Multiplayer Mechanic

Socializing costs and privatizing benefits is a lousy combination.

Many games allow you to increase your difficulty and your reward. This could be explicit in the form of a difficulty dial tied to rewards, but it is more often an opportunity cost. For example, you might equip an item that improves your loot, but doing so forgoes equipping an item that improves your damage. The fight is marginally harder and your rewards are marginally better. Kingdom of Loathing is an example of a game that does both: there are ways to increase monster level, and you can also equip items that have +monster level instead of (or in addition to) stat bonuses.

Kingdom of Loathing is also a single-player game. City of Heroes similarly gives you tools to adjust mission difficulty, and it gives the same difficulty increase and reward increase to everyone.

Multiplayer games that allow individuals to equip +loot items allow those individuals to increase their rewards at a cost of increased difficulty to everyone on the team. Alice is a tank using best-in-slot gear for damage resistance while Bob is a healer using best-in-slot gear for improved loot drops; Alice is working harder and incurring more repair costs for Bob’s benefits. Alice’s only way to avoid players like Bob is to stick with known companions or be That Guy and demand to see your equipment before letting you into the group. If everyone or no one is wearing +loot gear, the situation is fair and both risks and rewards are shared. Allowing individuals to unilaterally increase group difficulty for personal benefit is a solid example of anti-social design. Continue reading Loot Bonuses: Bad Multiplayer Mechanic

To-Hit Rolls

To-hit rolls are an RPG mechanic inherited from pen-and-paper systems. They represent an obvious intuition (attacks can miss) and use a binomial mechanic with a random chance. Many non-RPG computer games use a different mechanic: did the sword, shot, spell, or whatever hit the target?

One of the City of Heroes developers remarked that, had he to do it over again, he would not have included a to-hit roll or an accuracy stat. Every attack would hit unless some defense caused it to miss, and then you would have an indicator of why you missed. Continue reading To-Hit Rolls

Blast from the Past: Tome of Knowledge and Sets

I am still waiting for games to pick up this idea from 2009. Achievement systems have proliferated, tracking all kinds of things, but most games want to give you cosmetic items instead of unlocks. One specific item in that last post has been addressed by many games: a mount tab instead of making you carry mounts around. City of Heroes has always rewarded players by unlocking costume pieces, and Borderlands 2 lets you find/win/buy customization options.

With the upswing in F2P, however, life moves in the opposite direction. Storage space and cosmetic customization are ways they make money, so of course they charge you per item per change in appearance.

: Zubon

Estimating Difficulty

When A Tale in the Desert introduced barley as a growable crop, they also added a technology that could be unlocked by donating 100,000 barley to a university. How did they get the number 100,000? Nekhmet (one of the developers) grew a bunch of barley, they figured that the players would learn more efficient techniques (ATitD uses player skill-based crafting), and then they multiplied to get a large but not ridiculous number of hours of work. It turned out that Nekhmet was a prodigy at growing barley, at that technology was unavailable for months until ad hoc additions to the game allowed barley output to double and triple.

When Guild Wars 2 introduced pumpkin carving, a few hundred pumpkins were hidden around the world. It was an exploration achievement: find 150 to unlock the title. A technological problem let the same pumpkins respawn after carving, and they spawned on a per-character basis for a per-account achievement, so you could get the title without leaving Lion’s Arch.

When The Lord of the Rings Online introduced Mines of Moria, the dungeon fights that were its endgame were a mass of bugs and exploits, some of which were obviously unintended (stand in a doorway while a door closes: your weapons are on one side, your body is on the other, and the boss cannot hit you) while others surprised the players when they were declared “unintended” (kite the boss around his throne so that it is between the two of you when he uses his devastating area effect attack).

When City of Heroes introduced the Hamidon raid, players found a variety of ways to beat it, ranging from sniping it from beyond its range to capitalizing on teleportation and invulnerability to avoid damage. For months, every technique used was patched away as an unintended exploit. Some developers claimed that there was an intended way to beat Hamidon, but the players never seemed to find the “intended” one, and it is not clear whether it would have actually worked. Hamidon was later reconfigured into a fight with a more obvious “intended” approach.

Guild Wars 2 has a pop-up warning when you start the cooking crafting skill, telling you that it is more expensive in terms of time, silver, and karma than the other trade skills. Cooking is the fastest, cheapest, easiest craft to take to 400 skill, notably having the last points available for a few hundred karma worth of peaches where other skills require dozens of drops or even globs of ectoplasm.

Can you cite a dozen examples from your gaming history where “hard” content was trivial while “easy” content was literally impossible at release? Can you see why I am suspicious of any player claims about how hard something is supposed to be, what the developers’ intent was, or who this is for?

: Zubon

[CoX] Save Paragon City

The folks at CoHTitan are trying to keep City of Heroes alive. You can follow and support their continuing adventures here. An interesting item developed is the Sentinel+ Extractor, intended to copy character data for potential reconstruction, say in a third-party emulator.

If they can pull it off, I am happy for them. Having played my years in the City, I find myself unlikely to support the game financially, so I cannot dispute the NCSoft business decision. In meatspace, I am used to seeing people support companies, downtowns, mom & pop shops, etc. verbally but not financially. It is a different sort of inverse of Kickstarter. If the dollars can be found to make CoX a viable proposition, onwards!

: Zubon

[CoX] City of Heroes Closing

City of Heroes will go offline in a few months after 8.5 years, two expansions, and 23 issues (major updates). I played for about half that time, level-capping 11 characters, getting my money’s worth, and finding examples I still keep citing under the heading, “Congratulations on catching up to CoX in 2004” (or “…ATitD in 2002”).

Ethic’s 2005 post about the end of Asheron’s Call 2 remains one of the most popular pages on the site.

: Zubon

[GW2] Scaling

SynCaine is right: Guild Wars 2 does not scale perfectly when you visit lower-level zones. As you might expect, a level 30-something scaled down to level 4 is still more powerful than a level 4, and then the content was balanced around level 2-3. I confirmed this when I took a low-30s elementalist into Queensdale and one-shotted elementals with Arcane Blast. Fast-leveling guildmates also report feeling much more powerful in WvW as a 50 scaled up to 80 than as a 10 (also scaled up to 80); this reminds me of WAR scenarios, where you knew you were going to lose when half your team queued up a the minimum level. Scaling does not overcome the fact that higher level characters have more and better gear, traits, and abilities.

Some of that may be a factor of the small numbers involved. This is not hard math: 2 is half of 4 but only 10% of 20, so a two-level difference means a lot more at level 4 than 20. Visiting areas in the teens and 20s, the difference seems less severe, although I have not tested it vigorously. I also expect there to be some scaling tweaks in place by the time we have enough level-capped characters to test scaling over its full range.

As usual, I would like to point out that City of Heroes solved this problem years ago. Your entire group is set to the same level. You can set the content to whatever level and group size you like. Here you would scale the characters rather than the content, but the principle is the same. City of Heroes faces the same problem, that a level 50’s enhancements make him/her far more effective when scaled down to 20 than a level 20 character is, but you can always choose to adjust your challenge level if you want a more authentic level 20 experience.

: Zubon