The Unfortunate Obsession with Metacritic

The industry has a bit of an obsession with Metacritic scores. Both consumers, and unfortunately publishers, look at the numbers in a way that has proven not only counterproductive, but dangerous in recent years.

 

Metacritic (And similar aggregators) have a simple job: Collect review scores and average them out. Now that’s all well and good and can be a useful source for a range of different reviews on a game, movie, music and so on.

Metacritic though has become a focal point. The vaunted goal, the barometer of what’s worth buying, and even what determines developers getting bonuses. Yes, Metacritic alone has become a huge part of the industry, and while it does a job that is needed, namely collecting reviews into one place for convenience, how it does it and the impact of that, is the problem.

Metacritic

First, the rise of competitors such as OpenCritic have raised awareness that Metacritic seemingly acts selectively with getting reviews from places, requiring verification. Further, it also reaches very slightly different averages, indicating that some reviews are weighted more than others. Weighting is in fact a key point that we will come back to later.

Another issue is review scores, because that’s what Metacritic uses for an average, will be based on different criteria. A 7 from one site is a different set of criteria from a 7 elsewhere. But for Metacritic, a 7 is a 7. The reasoning behind the number could be completely different, but the context behind that number is ultimately lost.

The idea that Metacritic and other aggregators can give a consensus is a bit foolish. ure, it will say “Generally Favourable” or “Mixed” or whatever other word of the times it chooses, but is that accurate, when the context behind those numbers is lost? The average number is just a basis from generalisation.  A game could come out with 80/100 and be “Generally Favourable” only for the reality underneath to be…well mixed. This is especially prevalent with mixed or divisive games, as outliers skew the data, and in statistics, significant outliers are anomalous. But Metacritic doesn’t care about anomalies and their context, just the number. More pull is assigned to the lower end of the scale, so even if a game has by all accounts more “Positive” reviews than “Mixed”, the “Mixed” weight it down. On top of THAT, not every game will have the same number of reviews, further skewing data.

original

 

I’m going to keep bringing up that number, because just like on this site and our refusal to score reviews (Again, context is key), that number in recent years has become the most contested aspect of any game.

The 4 point scale, admittedly nowadays more like 3 point scale, is a problem. Undoubtedly so in fact, to the point where now, a game below an 8/10 is considered bad. Yes, really, and I wish it wasn’t the case. A key factor in this is in fact aggregators have weighted (There is that word again) the averages.

Say you have a 10 point scale, in this case 1-100. If that was to represent a range of values from terrible, to bad, to average, to great, to excellent, why would 50% of that scale be assigned to “Negative” and below? Why is 50%-75% the range for “Average”? Why is the range for “Good to Excellent” only the upper 25%?

MTIyNDU3OTcxNzE1MzEwMTgy

This doesn’t make any sense. The cream of the crop would stand out regardless, so why is there a significantly larger range for games to be considered bad, than the other 2 general ratings? Why is the general bar for “Okay” around, of all things, 75-80%?

This extends to the issue regarding the 4 point scale. As the averages are locked to this upper half of the board, and most games fall in that range, it starts to push the minimum for what gamers call “Acceptable” up and up over time. Once, it was 7. It’s closer to 8 now and heaven forbid it hits 9/10.

These scores, and the uneven distribution and attribution of values given to the scores, is simply nothing more than fuel for a fire, of my game is better than yours, and so on so forth, amongst gamers, or even attacks on developers. It’s okay to have high standards, but average is not 7/10.

untitled5

Finally we are going to come to the other obsessed group: Publishers. Companies like EA and a handful of others, are known for tying bonuses for developers to a Metacritic score. Get a certain score, or no bonus. This is the dumbest, most disrespectful thing imaginable.

Sure, you should get a bonus for doing a good job or working extra hard. That is true of any industry. But developers go through crunch time, unpaid overtime, without union support. Worse to that, if the game has great visuals, and those artists, animators and modellers did the best they have ever done, but the programming leaves the game a mess with a low score, those visual development staff won’t get a bonus, even if it is their best work.

This is of course, assuming they still have the job afterwards, due to high turnover in the industry as well. Bonuses for a developer should be based on the work of the individual, and not held behind an arbitrary and without context number that a publisher wants to see, that can be broken down by a completely separate part of the development staff.

 

 

The industry has manifested a culture of abusing developers and not giving them their dues, based on what other people think. Not their work individually, not even what the publisher thinks, but what the rest of the world thinks, and the obsession with every increasing standards and a shrinking scale of what is acceptable means that this culture will only hurt developers in the end.

 

As always if you enjoyed this, give it a share and let me know what you think on social media. Until next time, Happy Gaming!