Testy

Pvthudson would like you to test the game if you are a beta tester. Revolutionary concept, I know. I skip most betas for that reason: why provide free bug-testing for some company when you could pay to play someone’s live-but-should-still-be-in-beta MMO? Wait, that didn’t come out right…

I propose something in the opposite direction: developers, please tell your players what you want from this test. Pvthudson cites the mis-labeled stress test. If you are not seeking feedback on something, let us know that, since it does no one any good if you invite 20,000 people and they fill your forums with comments on balance. And frankly, it will just annoy everyone who feels like his feedback was ignored. On your test server, please do not describe the change as “increased fire damage”; your players cannot test if something is working as intended unless you tell them what is intended. Don’t be vague or coy here. You can even tell players, “This will be on test until we are sure of X, Y, and Z.”

If you want to harness the power of your playerbase, you should attach that harness to something.

: Zubon

4 thoughts on “Testy”

  1. Here is a direct shot at a common beta problem:

    HAVE AN ENTIRE BETA FORUMS, NOT JUST ONE SUBJECT ON YOUR MAIN FOLDERS.

    I despise having to sift through one folder for information instead of having a mirror of the main forums broken up by class, skill tree, gameplay etc.. Have a first impressions thred. Have a bug report thread.

    DO NOT MAKE ME DO SOME CRAZY BUG REPORTING PROCESS. Make it simple to report bugs, or people will just be lazy and not report them. A recent beta I was in had such a convolouted reporting procedure, that I recorded an entire notebook of bugs (it was pretty buggy) and then never submitted the reports because it took me so long to report the first two.

  2. I’ve been doing a few of these for several years, and the thing that strikes me about them seems to be about the “open betas” isn’t bug testing that normal beta should be, but more for weeding out exploits and balance type things. Get people to play your game in ways you haven’t thought of and see if they can do things you didn’t intend. If you are trying to find bugs, that should be your intent, not using it as a demo. Nothing is more frustrating than being able to repeat a bug each time you try it, reporting it, asking if anyone else can do it, being told “L2P”, and still seeing the bug in the live game months after release. There should be a better term for what open beta or stress tests really seem to be, it would be a bit more honest i think.

  3. There’s a few ways of looking at it, I guess.

    One, are you testing for bugs? In that case, maybe you should have an incentive program for reporting them (one free week of play when the game goes live per verified bug reported?) rather than just invite a whole slew of people in to get annoyed by all the bugs and tell all their friends about how the game is crap and unplayable.

    Two, are you testing for exploits? In that case, maybe bug reports don’t concern you overly much – you just want people to play the way they usually play, and then mine the database for any sign of exploiting. It might be a good idea to ask people to try to game the system if you really want robustness.

    Three, how about just being there and asking people nicely? In the Dungeons and Dragons Online stress test, the devs were sitting right there in the tavern, chatting away. At certain points they would ask people to please come to the tavern since they were testing server load. Once there were enough people there, they kept us there by running trivia contests with real prizes like beta spots and video cards.

    Four, I think “open beta” has become more a mandatory stage of game promotion than actual bug-hunting in recent years, which is kind of sad and pathetic.

  4. I think his post was overstated, might have held better if it was regarding a true closed beta or alpha where beta testers from the public were expected to provide detailed error/bug reports.

    The reality of today is these beta’s are used for stress tests, monitoring of exploits, marketing/promotional efforts, and progression checks. Very few have any requirement or expectation of public testers to report on bugs.

    The reward program is popular for internal teams, and I’ve seen it work well for public beta’s too.

    The whole lesson of this is if Dev’s have an expectation of actual beta testing, that needs to be made extremely clear, and those testers that aren’t submitting even a single quality report every month or so would be removed from the beta and a new tester let in.

    Kinda hard to fault people for treating a MMO beta as a trial when they paid a 3 month FilePlanet.com subscription in order to get it y’know?

Comments are closed.