Back in the 00s, there was a fashionable notion called “The Wisdom of Crowds” – the idea being that, if you solicited enough people’s answers to a given question, you’d get an accurate answer. But with book or film reviews, this is demonstrably not true. Rather than wisdom, you get the tyranny of the majority – a mob that knows what it likes and likes what it knows. Any informed opinion is drowned out. Which is why on Rotten Tomatoes (which aggregates critical opinion), Transformers: Age of the Fallen gets a 19pc rating while on Amazon it gets a nice fat four stars.
OK, you might say, but you’re being a snob. Maybe my not liking EL James or Transformers films doesn’t mean they’re bad. It just means they don’t appeal to my hoity-toity elitist tastes. As The Dude says in The Big Lebowski, “That’s just your opinion man.”
But there’s a problem with this argument. Once you start conflating popular with good, you inevitably embark upon a road to all sorts of fun conclusions like McDonald’s being finest restaurant in the world, Dan Brown being one of our great authors and Benidorm being the very best holiday destination there is.
In fact, I have a feeling that Amazon reviews have actually got worse over the years. The reason? Back in the Amazon’s salad days, its reviewers were likely to be early adopters who were better-read than average. Now, they’re more like a cross-section of society and the crowd isn’t so wise anymore. I know, know: you’re not supposed to say these things. But then you look at the thousands of five star ratings that Jeffrey Archer gets – and what else is there to say?
But even if you are a Dan Brown fan, you may still get a raw deal from the ratings system. The recommendations system based on your own and others’ browsing is unlikely to serve you up the mid 80s Hungarian literary classic Sátántangó (“Not a plot driven piece. It moves at the pace of a stone hedgehog.”) on the basis of having bought The Da Vinci Code (3.5*), but it might give you something like Norwegian by Night (4*) or The Laughing Monsters (3.5*). Both of these are classed as thrillers, but they are of the thoughtful, demanding kind. Which is why you end up with Amazon reviews such as “The more I read the more confused I got” (Laughing Monstors) and “So much E – L -O – N – G – A – T – E – D detail, EXTREMELY BORING” (Norwegian by Night)
Still, I dare say both are fair reviews if you’ve picked your 3.5-4* thrillerexpecting zingers like, “My French stinks, Langdon thought, but my zodiac iconography is pretty good.”
“Once you start conflating popular with good, you inevitably embark upon a road to all sorts of fun conclusions like McDonald’s being finest restaurant in the world”
There’s more to it though. The system itself seems designed to bunch reviews around the four star mark. You can’t award a product zero stars. The minimum is one. But the fact is most people give good ratings. A helpful 2014 analysis of new electronics products showed that over half the people rating the products gave five stars. The next biggest was four. The average is 3.9.
It’s not just Amazon either: FiveThirtyEight says that film review sitesthat rely on the public suffer from the same highly positive bunching.
Back on Amazon, and this means that most products get a rating between 3.5 and 5, so you effectively have four different ratings (Amazon rounds to the nearest half star). In the abstract, you may understand this, but in practice, your brain tells you that there’s very little between a 3.5 star product and a 4.5 star product. Except, when you’re using this compressed scale, the former effectively is 25pc and the latter 75pc.
This wouldn’t matter if the raw material for the ratings wasn’t so dodgy, but it is. Let’s take a film – Under the Skin. It gets a mere three stars out of five on Amazon – and the largest single rating is a single star – which, pretty much puts it down in “It sucks” territory. Yet quite a few professional critics rated it one of the best films of 2014.
I suspect Under The Skin suffers from having a big name (Scarlett Johanssen) and a well-known director (Jonathan Glazer). Like Norwegian by Night and The Laughing Monsters, this put it in crossover territory where, despite being a difficult, arty film, it gets a fair number of Transformers: Revenge of the Fallen fans. They watch it and, perhaps, unsurprisingly, think it sucks. So the superb Under The Skin gets three stars while Transformers, which really does suck, gets four.
None of this is going to stop me from using Amazon, but it does make me wonder if, rather than suing the shills, Amazon’s time and money might be better spent funding professional critics. As it is, all the current system does is provide an answer to the question, “Is it better to ask the opinion of one person who reads 100 books a year or 100 people who read one book a year?”