Nowhere is the Internet's democratic nature clearer than in the user-generated reviews on such websites as Yelp and TripAdvisor.
All it takes to weigh in on a restaurant, hotel, spa, bar, bowling alley — anything, really — is a user name and password. Although they're a boon for consumers sharing unfiltered ideas and opinions, there are obvious hazards: What can be trusted? How do you know a hotel owner hasn't put family members or friends up to clogging sites with positive reviews of his property and negative reviews of his competition?
And what should you make of scores that are spread all over the board? Consider, say, the Holiday Inn Select in downtown St. Louis. It recently had nine "excellent" ratings, 25 "very good," 26 "average," 18 "poor" and 19 "terrible."
How can 34 people rate a property above average and 37 say it is below average? Though such sites can be helpful when you're traveling, more than once I've wondered whether they can be trusted.
Jonathan Barsky says yes, at least when it comes to TripAdvisor. Barsky, co-founder of Market Metrix, a firm that tracks customer satisfaction in the hospitality industry, compared reviews of 67 hotels on TripAdvisor with scores derived separately by his Market Metrix Hotel Index. Market Metrix and TripAdvisor have an information-sharing partnership, but "the study had nothing to do with the nature of our relationship; it only enabled the study to take place," Barsky said.
His conclusion was that the scores generally matched up, though TripAdvisor ratings had higher highs and lower lows. But the conclusions and general sentiments were largely the same.
Barsky attributes the unpredictability at the edges to passion. People often submit ratings only after exceptional experiences, whether good or bad.
I suggested that there also could be foul play, and he agreed, noting that little can be authenticated, meaning whether someone actually stayed at the hotel. "In any open system like that, you have to expect that," he said.
The solution is to use such sites critically. Read the most favorable reviews and the least favorable, but don't take them as gospel. Pay more attention to the middle and, perhaps most important, look for trends.
"Most hotels don't have that many reviews on TripAdvisor, so it doesn't take that long to read them all," he said. "If a bunch of people say the pool stinks or the workout room is as big as a closet, I might look elsewhere."
Ultimately, he said, perception is a function of expectation.
"It's based on what you get for your money," Barsky said. "People can be disappointed with the Four Seasons or Ritz-Carlton."
Adam Medros, vice president of product for TripAdvisor, said the size of the website — along with a team of about 25 workers dedicated to "content integrity" — protects its accuracy. He also suggests forming opinions based on multiple reviews, rather than just a few.
"The owner [of a hotel] and their friends will never be able to write enough reviews to tip the system," he said. "The sheer number of travelers will outweigh that. The system will heal itself."