For those who aren’t aware, the new SimCity title is having a huge number of problems. A number of these problems are caused by the always-online requirement; with servers being slammed and other bugs interfering, players are finding themselves locked out of the single player experience a lot of purchasers want.
I was eyeballing SimCity review scores on Metacritic as a point of interest to see how different sites scored SimCity. A glance over these scores turned up something interesting: it seemed like all the sites that got early previews and most likely played on EA’s private servers (as Polygon did) gave the title much higher scores. To see if this was true, I charted SimCity’s professional Metracritic review scores over time and this is the result:
SimCity (2013) review scores from March 4 to March 17, as indicated on Metacritic. (Click to enlarge)
It’s pretty clear that the day 1 and day 2 reviews from gaming press sites were much, much more positive than any site that reviewed it later on. They cluster up in the 80% to 100% range, but even a few days later the review scores are much more spread. Polygon has three review scores up there to reflect the changing review scores it gave SimCity – a gushing March 4 review gave the title 9.5, a day later that was revised to 8.0 and then on March 7 it was downgraded again to 4.0.
So, to things that come to mind:
- Those gaming
journalistswriters that got early access to SimCity and most likely played on EA’s private servers did not get an accurate view of what SimCity would be like to play at launch. The result of this is that the Day 1 reviews would have encouraged people to purchase a title that they may have been locked out of due to server issues. - Reviews posted later have had the benefit of seeing the situation blow up and possibly experienced the frustration of not being able to log in themselves. However, but this time the controversy was in full flight, so it’s very possible that the reviewers tempered their scores knowing how the masses were reacting to those issues i.e. not well.
One thing that might blur the line here with some of these scores: we don’t know when the actual reviews were written. Just because they were published on a certain day doesn’t mean that they weren’t written and finalised well before that date. It would be interesting to know if those who returned the higher review scores had played it pre-launch on EA’s servers or had actually had a good SimCity experience after the title had started its retail sales.
Regardless, it’s a concerning issue when reviewers are experiencing a completely different game than players will and then recommending the title based on something that may not be replicated after launch.
World Class Villains 
With few exceptions, the revenue stream for gaming “writing/journalism” is ad views on your content – the actual consumers of said content rarely pay and arguably get what we collectively pay for. If you’re in the business of chasing ad views then your content needs to be timely. If the cost of being timely is basing your review on pre-release access that may not represent the final product… well, at least you get more ad views when people re-load your article to see how many more points you removed today. Meanwhile, if you are coming late to the party and you can already see which way the wind is blowing, which approach will get you more hits – making people feel smart by confirming their opinion that the launch is a cluster or challenging them with a contrary view?
Note that I don’t put us bloggers on any higher of a pedestal. Ever notice how no MMO ever does an extended period of open non-NDA testing anymore? How betas are now carefully managed weekend events designed to concentrate the word of mouth and keep anyone from getting high enough in levels to get a feel for what the endgame (if any) will be like? None of us who write posts about these managed access events are any more fit to throw stones at those of us who are sufficiently enterprising to get paid for writing it.
Some sites did that with SimCity – release a ‘pre-review’ and then another review when the game had been out for a bit. As you say, it’s double the content to sell advertising on.
MMO betas have definitely become weak events – I’m not sure how much testing actually can be done during the limited beta windows, especially if not all the content / systems have been released yet. And yes, bloggers are just as easily influenced. Especially if they are wined and dined as several articles where a blogger was taken on a studio tour have shown.
Pingback: Are Day 1 Game Reviews Positively Biased? Short Answer: Yes | Vicarious Existence