Buyer Beware: Yelp’s Sorting of Reviews Misleads Readers

Posted on
4 min to read

No matter how good a business’s product or customer service is, you’ll likely find some negative reviews.  Some would say even more so on Yelp than other sites.  The bigger the business, the more total reviews they’ll have and likewise the more negative reviews they’ll have.  At the bottom of nearly every business’s profile page on Yelp you’ll find a link to all the “not currently recommended” reviews, previously known as “filtered” reviews.  Currently, Montway has 36 recommended reviews and 215 not recommended, both types have an average rating of 4 stars.  (We have many more 5-star average reviews on other sites, like here and here. More than most auto transport companies but that’s a topic for another day.)

If you take a look at the “not currently recommended” reviews for any business, you’ll likely be surprised at just how negative they are.  I certainly was.  If you look a little closer, you’ll eventually notice that Yelp is choosing to sort these reviews first by lowest rating, then by most recent.

Case Study

At first we thought this was a bug or somehow unique to our page, but after looking around, we saw it everywhere. Take this example, for what is arguably the best restaurant in Chicago, Alinea. They’ve got 1,121 featured reviews with an average of 4.5 stars.  However anyone who views the not currently recommended page would certainly get a far different impression.

The average Yelp user who happens to click this link at the bottom of the page (for any business) most likely will not figure out how Yelp is sorting the reviews.  That user could quite reasonably come to the conclusion that all of these supplemental reviews are negative or that this is where all the negative reviews are hidden. The reader might even think that’s the sole reason they’re not recommended, but that they are still somehow valid.

They might think Yelp is actually trying to protect the business by putting all the negative reviews in this section and not on the previous page.  Nowhere on the supplemental page is a distribution of star ratings, which is available for featured reviews by clicking “Details”.  Also nowhere on this page is an explanation as to how reviews are sorted nor why.  It only mentions that “the reviews below are not factored into the business’s overall star rating.”  Their FAQ has a section about recommended vs. not recommended reviews, but no mention of sorting.

Anyone who happens to click the link at the bottom of the profile page would find it hard not to have a lower opinion of the business after seeing the page.  In some cases it may not change their willingness to patronize the business, but it would be very hard to imagine it doesn’t impair their impression of the brand at some level.  For many businesses, this shockingly negative impression could be enough to drive the customer elsewhere, especially considering that people tend to “overvalue” their most-recent experience or information.

The implication of this sorting method is that Yelp will show every single possible 1-star review (not currently recommended), going back to the beginning of the businesses history on Yelp, then showing every single 2-star before finally showing the first mildly-positive review.

I would argue this is completely unfair to the business and is also a disservice to Yelp users by not accurately representing the current customer experience they could expect at a business.   A business could have hundreds of positive not recommended reviews which get buried by a relative handful of negatives.

If these “not currently recommended” reviews are of questionable value in Yelp’s eyes, why show them at all?  If they do choose to include them on a separate page, why not use the same sorting method used for recommended reviews or simply sort them newest on top regardless of rating?  Why not at least include a graphic representation of the distribution, i.e. how many 1, 2, 3, 4 and 5 star reviews?

We brought this issue to our Yelp Sales Rep.’s attention and got the following response:

 “Thank you for your email and your expressed concern.  Recently, Yelp rebranded our automated software from “The Filter” to a more appropriate name of “The Recommendation Software”.  A wide held belief from Yelp users and businesses on Yelp was that the filter only filtered out positive reviews from businesses.  The switch to sorting them by star rating was very strategic on our part.  The reason behind it was to debunk the held belief that the recommendation software only doesn’t currently recommend positive reviews.  They are now sorted so that businesses and Yelp consumers are able to see that the currently not recommended reviews section will contain reviews of all standings.

Let me know if you need any more further clarification.”

In the past, businesses have complained that all their positive reviews seemed to be filtered and negative reviews disproportionately shown, among other complaints. It sounds to me like Yelp is using this as pretext, in effect saying ‘Look! There’s negative reviews filtered too.  We’ll even put them all at the top so you don’t miss them.’  To be clear, this is my interpretation, not their quote.  One could read further into Yelp’s statement and my interpretation of it to get the impression this is Yelp’s way of ‘sticking it’ to the business community that has given Yelp so much bad press in the past. The initial complaints (whether true or not) were that all the positive reviews were on the filtered page.  Yelp is acknowledging this perception (while not admitting it’s validity) and the response is in effect ‘now the supplemental page won’t have positive reviews either.’ Again, my editorializing.

Just to be sure, we asked for clarification that we were indeed both talking about the same thing and again got the following response:

 “Thank you for your email follow up.  At this time, Yelp is standing behind our decision to list not currently recommended reviews in the order that they are.  As I stated, this showed that the recommendation software is not biased.  It also shows that there is a reason that these reviews are not currently recommended, so I do not agree that this would look negatively on your company by a user.  the reviews are not recommended and therefore are not held as highly as those that are recommended.

Ultimately, at the end of the day, this is a site feature that we are not able to control.  If there is anything else I can help you with concerning your Yelp listing and your advertising that is outside of reviews, let me know!”

But wait, it gets worse.  As if showing the 10 worst not currently recommended reviews isn’t enough, Yelp then shows you the 10 worst “removed” reviews with little more than the 1 star rating.  What is the purpose of showing what most would argue is the most meaningful part of a review (the star rating) for reviews which were removed for violating Yelp’s own content guidelines or terms of service?  Anyone scanning the page will first be attracted to the images (1-star ratings) and may not look too much into the details (hey, this review is removed – why is it still here then?) Many users will simply see a whole page full of 1-star reviews.  The inclusion of these additional negative reviews really starts to question Yelp’s motivation for presenting this page as it does.

For Alinea, Yelp’s negative reviews likely won’t cost them any business (they are booked up well in advance and have only so many seats) but most other businesses wouldn’t be so lucky.

In researching this issue, there didn’t seem to be anything out there on the topic, hence this post to call attention to the issue. Hopefully with some others asking similar questions we’ll see some (positive) changes from Yelp.

Get an instant quote