How To Use the Test Server

Fix the major bugs that are found. Test for both bugs and balance. I suppose these both could be filed under “listen to your testers.” It is fine to release live with some known minor bugs; there is a trade-off between delaying content and making a more perfect union. We know that we will need to fix some things after the patch.

I know that changing something by 20% is just a minor tweak to the code, but it makes a pretty big difference if you do so after the new content has been out for several weeks, and it looks bad when players were pointing the problem out on your test boards since the day it hit the test server.

Internal Q&A is probably very good. Few things make it to the test servers completely and utterly broken on any game. The goal is to see what happens when 10,000 people all bang away at it. Your internal team could not run through each part of the content 100 times, so this is desirable.

That should not be the only purpose of the test server. Yes, it is a small and unrepresentative sub-sample of your player base, but if someone is willing to put 10-20 hours a week into testing the new stuff, you might want to hear his conclusions about how it plays in the field.

You also might want to give players boosts of various sorts to let them test effectively. For example, if your players need to spend 50+ hours leveling characters on the test server to test the endgame content, your endgame content is going to be tested pretty badly. Perhaps this is why non-bug report feedback is often set aside: two wrong make a “meh it doesn’t matter.”

Say you release content and balance it upwards later, by which I mean buffing the new class, skill, power set, whatever. Who will react more strongly: the people who are happy about getting a bonus to their new class, or the people who are really bitter about having ground out 20 levels with the gimped version? There will be posts thanking you, but what about the rather lengthy threads discussing the issue for preceding weeks? After a month on test and three thousand-reply threads about how the new class is too weak, it goes live as is, but gets changed two weeks later? Did something change in those two weeks, or are you trying to give players the impression that their test feedback is unimportant?

Personally, I no longer play on test servers. It is close enough to paying to beta on the live servers.

: Zubon

3 thoughts on “How To Use the Test Server”

  1. I feel your pain. After numerous betas… as well as being a dedicated test server resident on many games, I’ve come to the conclusion that the majority of MMOs dont really listen to tester feedback. Horizons would be a perfect example. They had more bugs than I had lead in my pencil to write down. Every single one submitted, and every single one went live.

    AO (at least for a while) seemed responsive to it’s test server population. Once the Devs would listen, it was like the game improved by leaps and bounds. Sadly, that is rare.

    I’ve often wondered the purpose of having an open test server, if you will not listen to the feedback people give you…on their time, as the test server time in no way advances their ‘real’ character(s).

  2. I played on EQ’s Test server almost my entire play career. I would say, without question, that that was at least one reason I continued to play. I was priviledged to have access to the devs, more than the average Test resident at times even, and in the last year or two there was a lot more interaction of what needed to be tested, which really helped us feel like we are making a difference. Also, the dev staff was more receptive to ideas, and also were much more outgoing.

    Flashing back to my DAoC career, I also spent a lot of time on the Test server, working with the devs. I had a bit of a falling out with them, as at this time they were still on the “we can do no wrong” mindset as they were stealing players from EQ at a steady rate. A lot of friends who had also left from EQ Test to DAoC Test left around the time I did, for general disatisfaction as well. Neither game gave any props at all to their public Test residents.

    Fast forward to today, and today’s EQ Test server people are getting some love, although in my opinion, it’s too late to help and still misdirected. Things like all veteran benefits, double exp, extended Fableds, and other perks are nice, but they fail to reward a person who actively bug hunts. All they do is make the server have a faster mid game, and help the mostly casual player base grind AA. There needs to be some sort of mechanism in place to reward those who help find bugs, and even more communication about what was changed. The fear of their proposed bug-fixes being posted often presents them from listing them out. What the inevitable end result is that they end up found by players instead and complained about. A proactive stance would really look better. It remains to be seen if they can attract true testers to the server and still keep old-time faithful testers happy.

  3. My “test server” experiences that have actually involved a dev or two listening to what the players had to say have been limited to ATITD. In the original run, I ran up a mountain and ran for an hour and a half along the peak of the range before learning that there was no way down. Quick hail to the lead developer and, despite the fact that this area was little-traveled, he both transported me down the mountainside without me losing progress and went back to make sure no one could get up there in the first place. I really think more developers should take the quasi-GM approach to bug/design reports. It works.

Comments are closed.