Yelp confirms Harvard study about fraudulent reviews, says its algorithm discards 25% of user submissions

Earlier this week, a Harvard Business School study (PDF) titled “Fake It Till You Make It: Reputation, Competition, and Yelp Review Fraud” found businesses that don’t have a good reputation online try to “fix” the problem by submitting fake reviews. This was quickly misconstrued by some publications as a claim that the percentage of fake reviews featured on Yelp is growing.

A quick look at the study, however, shows that this is not in line with the findings. In fact, Yelp today confirmed the study is accurate and noted its algorithm, also referred to as its review filter, only selects about 75 percent of the service’s 42 million contributed reviews:

This means about 25% of the reviews *submitted* to Yelp are not published on a business’s listing or recommended to consumers. Among these reviews are the ones the Harvard study found likely to be fakes submitted by businesses themselves.

We’re not sure where the confusion started, but this sentence in the conclusion may have done it: “We show that the problem is widespread – nearly one out of five reviews marked as fake, by Yelp’s algorithm.” This was twisted to say that 20 percent of reviews on Yelp are fake. As you can see in the quote above, that’s not true.

In fact, the paper’s authors themselves note that measuring the percentage of Yelp reviews that are fraudulent is not within their abilities. “Because one cannot directly observe which reviews are fake, we focus on reviews that Yelp’s algorithmic indicator has identified as fraudulent.”

The 💜 of EU tech

The latest rumblings from the EU tech scene, a story from our wise ol' founder Boris, and some questionable AI art. It's free, every week, in your inbox. Sign up now!

The paper’s abstract breaks down its findings more succinctly:

• Roughly 16 percent of restaurant reviews on Yelp are identified as fraudulent, and tend to be more extreme (favorable or unfavorable) than other reviews.
• A restaurant is more likely to commit review fraud when its reputation is weak, i.e., when it has few reviews, or it has recently received bad reviews.
• Chain restaurants – which benefit less from Yelp – are also less likely to commit review fraud.
• When restaurants face increased competition, they become more likely to leave unfavorable reviews for competitors.

In summary, a business’s decision to commit review fraud is driven more by competition and reputation incentives “than simply the restaurant’s ethics.” None of this is very surprising; if, however, a study comes out showing how many fake reviews slip past Yelp’s algorithm, we’ll be much more intrigued.

Top Image Credit: Spencer Platt / Getty Images