Five Stars? Really?

Consumers inflate overall ratings when asked to rate various aspects of their experience online.

accuracy-of-online-reviews

Many consumers check online consumer reviews before making a purchase, but gauging the reliability of the reviews can be tricky, according to U of A research (Photo: Getty Images)

More Canadians than ever are turning to online reviews before buying products or services.

According to a consumer website called Made in Canada, one in nine people will read at least one review before making purchasing decisions, and 33 per cent of Canadians say they feel strongly influenced by online reviews, the vast majority of which they find on Google, Yelp, Facebook and TripAdvisor.

Consumers are not likely to trust a company with fewer than four stars, typically filtering reviews for products and services rated four stars or higher. As a result, most reviews are bimodal, meaning they tend to cluster on the low and high ends of the rating scale, says Katie Mehr, a professor in the Alberta School of Business who applies experimental psychology to online consumer rating systems.

Gauging the accuracy of those reviews can be tricky, especially when an increasing number these days are fake. But one way to shed some light is to look at ratings of individual attributes, says Mehr.

In a recent study based on her doctoral dissertation and published in the Journal of Consumer Research, she found that when consumers were asked to rate specific attributes of a subpar experience (one with a mix of desirable and undesirable aspects), they inflated the overall rating.

“We find that consumers who experience bad service at a restaurant, for example, provide higher overall ratings of that experience when they are also asked to rate the restaurant on food, service, ambiance and value,” she says.

“Being able to rate a specific negative aspect of an experience makes consumers less inclined to incorporate that negative aspect into their overall evaluation.”

The same inflationary effect was not found when consumers rated individual attributes positively.

“We thought that perhaps if the food is bad at a restaurant, but you’re reminded the service was good, it would also lead to a higher overall rating. That’s not what we find.”

We find that consumers who experience bad service at a restaurant, for example, provide higher overall ratings of that experience when they are also asked to rate the restaurant on food, service, ambiance and value

Katie Mehr

accuracy-of-online-reviews
(Photo: Supplied)

Mehr conducted 10 experiments in which participants were asked to rate an experience described in detail, such as going to a restaurant, or rate a recent experience of their own, such as an Uber ride.

Once given the opportunity to air their grievance, participants wanted to “avoid negative redundancy” in their overall review, she says. In other words, the psychology at work seems to be, ‘Let me get this off my chest and then I’ll say something positive,’ especially since the norm in online reviews is to rate high.

“When people know they can voice the thing that went poorly — ‘Hey, the food specifically wasn’t good, or the cleanliness at this Airbnb wasn’t good’ — they weigh that less heavily in the overall rating as a result.”

The one exception to the rule is when a company offers both a rating option and a text box for further comments.

“They’re seen as two different ways of conveying feedback, so we think that people don’t feel like it’s redundant to be negative in both,” says Mehr. “It’s only when it feels like the same method of getting feedback that we get this effect.”

For companies trying to determine how consumers really feel about an experience, Mehr recommends offering only an overall rating. If they want to learn more about consumer impressions of specific attributes, however, the best approach might be to allow consumers to evaluate them while taking the inflated rating with a grain of salt — or perhaps using it for marketing purposes.

For consumers, Mehr’s study is one more thing to keep in mind when judging the accuracy of reviews.

“The idea of consumer ratings is really great,” she says. “You get information about a product or service that isn’t from the person trying to sell it to you — it’s from people like you. But it’s flawed in its execution in a lot of ways.”