The Time Isn't Now: Could Online Reviews Replace Customer Experience Surveys?

Today's second post comes from TMRE Platinum Sponsor Maritz. Author David Ensing, PhD, Vice President, VoC Integration will be presenting A Comparison of Website and Survey-Based Customer Comments: Do They Tell the Same Story? on Tuesday, November 13. Join David Ensing next week in Boca Raton and as a reader of this blog when you register with code TMRE12BLOG, you'll save 15% off the standard rate!

By: David Ensing

I saw an interesting commentary in Automotive News (an automotive trade publication geared mostly for dealers) titled, 'Do Online Reviews Trump CSI Surveys?' The basic gist of the article is that dealers are relying more and more on online consumer feedback to run their businesses, so shouldn't the auto manufacturers use this information as a replacement for their customer experience surveys? While I think that is an interesting idea, our research indicates it is an idea whose time has not yet come.

We've done studies in both the hotel and the automotive industries comparing online reviews to traditional customer experience surveys. As a matter of fact, Randy Brandt and I will be presenting the results of our hotel comparison at The Market Research Event conference in Boca Raton, Florida on Tuesday of next week at 2:45 p.m. If you are there, please stop in.

Not to steal our own thunder (BTW, here is a link to the apparent origin of that phrase) but presently we see lots of issues with using online reviews as a replacement for customer experience surveys. Some of these issues include:

• - There are systematic ratings differences between online reviews and traditional survey responses for the same hotels for the same time period. Interestingly, when just analyzing the comments, these differences are less pronounced.
• - The demographics of people that post online are different from those that respond to customer experience surveys.
• - Most online review sites do not, and cannot, verify that the person posting the review was actually a customer at the business. Online review sites in many industries are struggling with the issue of false reviews (both positive and negative).
• - Many online review sites allow businesses to 'manage' their reviews. Some allow businesses to intervene with the customer if a poor review is submitted before the review is posted publically.
• - Sample size is an extremely limiting factor when trying to use online reviews as a replacement for customer experience surveys. Currently, there is not enough information to develop reliable scores at the business-unit level based on review site information.
• - Even when there are adequate numbers of reviews for a given period, often those reviews are not representative of how customers are treated at businesses. Businesses with large number of reviews tend to be those that (wisely) pay close attention to how they are represented at review sites and actively manage their reviews. This is often done by steering happy customers to submit reviews but not steering unhappy customers the same way.

Given these and other issues, online reviews probably won't give companies an accurate view of their business-units' performance. However, it is probably still a good idea to monitor and understand this information because it is what customers see when making purchase decisions.

OK, I know what you may be thinking. Maybe something like, 'Well, that position conveniently supports the business he is in, doesn't it'? I agree, but I also truly believe it. Please post a comment or two and let me (and others) know what you think.

This post is co-posted with the SoundCheck Blog.