World Class Villains

The whole alpha / beta situation around MMOs is an interesting one. It’s a critical area of both testing and marketing for any MMO which is a curious place to be since ‘testing’ and ‘marketing’ are really at opposites ends of a gaming scale – testing is about fixing bugs and dealing with a rough hewn application that requires a lot of patience to deal with while marketing a title should be about showing the smoothest, prettiest experience in order to get people to pay money to keep playing. MMO betas have tended to deliver less-than-encouraging results in part due to the conflict between testing and marketing.

To be fair, you can’t only release a MMO to public testing (closed or open) and not expect it to be used by players in a marketing related way (i.e. decision whether to buy or not to buy and to tell other people what they know). People tend not to separate the two. Even the adamant ‘I only test not play, plus I file 100 bug reports per test’ beta testers still get an idea about whether they like a title or not from their testing experience. Those players who sign up to betas just to pay a game for free still provide data that tests systems and allows their refinement. You can’t do one without the other. (Also, from this point forward I’m going to refer to any kind of public closed alpha, closed beta, open beta etc as just ‘beta’, since this is the stage where non-employees of the company are invited to get involved and it is just too confusing to have so many terms for variations of the same thing.)

The reality is that what matters is how well you do each function, with the truth being that many recent MMOs have done both beta testing and beta marketing badly.

Obvious Mistakes Aren’t Always Obvious At The Time

A poor beta was indeed a painful experience for Fury.

The best example of a MMO that died as a result of a poor beta is Fury – in order to test it properly for PvP, they needed a lot of players, but the player churn on such an unpolished PvP game meant they had to keep inviting players, which in turn meant a lot of players saw a buggy, unfun mess. By the time it launched, a lot of players were already turned off Fury and it never generated enough momentum to survive. Perhaps that’s an obvious mistake, but I’m sure it is one that will be repeated again.

Another failure of beta in terms of testing and marketing can be seen in the more recent titles of Warhammer Online (WAR) and Champions Online (ChampO). Both WAR and ChampO failed to provide the game in its entirety to its beta testers. WAR’s beta tests tended to focus on specific areas with its testing population, so testing was concentrated and specific. That’s fantastic in some ways, but it completely ignores how the game would be played during ‘real’ conditions, such as when players are spread out over a zone / the game rather than clumped in one location. A fun beta experience – players involved in concentrated PvP over taking a keep – ends up not reflecting game play reality – flip a mostly undefended keep for the reward; wait until someone else flips it back and repeat.

A lack of testing time left Champions Online without a spark that players could grab onto.

Champions Online also failed to test their title under ‘real’ gameplay conditions due to short beta testing windows each week and content that wasn’t widely available to test for long before launch. Although the limited testing times helped concentrate players and check server loads, it was lousy at letting players experience the game in their own time. This means that when ChampO hit live, a lot of mistakes that were glossed over when players only get to play 4 hours a week suddenly started to shine with the servers up 24/7. The fact that ChampO came with a launch day patch that changed how defensive powers were balanced is testament enough to how beta wasn’t able to correctly work out the fixes that needed to be done (testing) nor resistant enough to accept a major game alteration (marketing) prior to launch.

WAR and ChampO did a poor job in both marketing and testing using their betas in that their testing didn’t provide important play data that would remain applicable outside beta. Poor marketing within and just out of the beta saw both titles launched straight into difficulty. I personally don’t think WAR is going to last beyond 2010 and ChampO will need to do a lot to bring back players that have quit.

Ultimately bad betas do a lot of harm to a MMO. Incorrect beta testing provides an inaccurate picture of a game that players communicate to each other, which ultimately backfires for marketing as the game released doesn’t match the game described. Of course, all this points to is that a MMO beta is a Catch-22 – you need players to test the broken version of the game so that a lot more players will be willing to play the fixed less broken one that launches; players that test a beta game get negative impressions that drive them (and others) away from the game, but without them experiencing those negatives they can’t be fixed.

The Easy Fix

Of course, there is no easy fix. MMO betas are expensive and any player who says, “Launch it when it is done” is ignoring that extended betas cost extra millions that may never be recouped if World of Warcraft decides it is going to release another expansion during your launch month. Publishers and developers aren’t bottomless pits of money who launch incomplete MMOs just for the fun of it. However, there are certainly things that a good beta has over a bad beta:

1. Extended beta testing periods – 24/7 or similar. A MMO needs to be played under as close to real conditions for players to see if the game is really fun or not. Stress / concentrated testing has its place, but that should be for major events – how a MMO plays on a day to day level is vital for a beta to test.
2. All major systems in and tested adequately before launch. Implementing your craft and mailing systems a week before open beta is not ‘adequate testing’. A dev might get away with implementing a new item naming system or something similar before launch, but that’s a ‘nice to have’ rather than a core selling point on the front of the game box.
3. Communication between devs and players. Players can’t mind read developer intentions about why or how a certain system works. Developers need to be able to explain why a system works as it does, how they expect the player to use it and then be able to take feedback and improve the system based on player comments. Sometimes developers will need to choose between the vision or player demands.
4. Good bug recording systems and bug squashing response time. Players who see the bugs they report being fixed end up being much happier than those who feel ignored by the developers.
5. A lack of hype. Players come into testing with expectations of the game that can be over-inflated, feel disappointed when those expectations aren’t met and then go off to tell all their friends about how bad the game is.

A good MMO can launch from a bad beta and a bad MMO can launch from a good beta, but ultimately a good beta should both improve the game and create word-of-mouth buzz about the title, which can only be helpful in such a competitive market.

Launching a MMO isn’t easy and players aren’t as patient as they once were. Having a good, solid beta that sees players both help to knock out the lion’s share of bugs while also walking away and evangelising the title to their friends is going to be increasingly important for any new title that wishes to make a name for itself, outside of buying a major IP that has an inbuilt fan base.