TMRE 2010: House lines or land lines? REI takes a look

by Bill Weylock, Brand3Sixty -

REI determined that the rising percentage of cell-phone-only users makes land lines an unreliable sole-source approach method. Then they run into the problems with cell phone: more expensive to source and reach, require an incentive, not location based, ratio of cell phones to landlines not known, not able to cross off a household after reaching a number (as you can with a landline).

So on to online sampling: guess what, Anne-Marie cites the GRIT 2010 study indicating a large percentage of research professionals think online sample is less reliable than the market generally appreciates.

Fascinating study by Washington Mutual indicates that respondents who take 5 or more surveys annually are less likely to desire financial products than others. Survey experience alone could account for up to 7% variation in their results. This suggests applying scoring algorithms to sample including, among other things, extent of participation in online surveys. Panel providers take note.

With problems in every conceivable approach, what to do?

REI does an annual brand health study using RDD with landlines... historically no stratification.

This year did 7 markets online, 2 markets by cell-phone at same time as normal landline fielding. Cell phone interviews were only n=35 in each market.

They found striking differences in brand awareness between landline and online methods, also in ad recall and % of respondents who shop REI.

Online figures were higher for online

Demographic comparisons revealed more Causcasians and more college graduates.

Online panel sample seems to be a difference audience from a landline sample. Perhaps a panel rewarded by retail incentives is more aware of and engaged with retail. Not projectible for REI.

Comparing landline and cell samples, awareness, recall and penetration of REI shoppers showed little difference.

Demographics suggested a reach into populations hard to reach by landlines.

Next year REI is going to test ABS with landline and cell phone data collection. They can match sample to cell or landline numbers at about 65% - 75%.

REI has rejected online sample and is harking back to a previous era in bringing ABS to the fore for 2011. It is an evolving experiment, but they expect that ABS with sampling shared by cell and landline phone outreach should provide more representative sample than previously obtainable.

Okay, online sample providers ... I did ask.

I asked Anne-Marie whether she was saying that online sample is unreliable and unprojectible and should be eschewed in favor of phone. Didn't come out that way, but that was the gist. I wanted to know whether there was something peculiar to REI that made the demographics of online sample unrepresentative of their universe.

Unwilling to go that far, Anne-Marie simply restated her opening thesis: that marketers should ensure that their sample is sufficiently representative of their markets.

I also asked, just prior to that, why she had been unable to weight the online sample to census. The rub seems to be that the panels were unable to provide sufficient penetration in key markets.