Response rates
Are you giving research responders what they want?
For the most part, researchers live off of surveys. The vast
majority of researchers specialize or have basic skills in writing surveys and
analyzing survey data. But in the last few years, many more tools have become
readily available to us.
majority of researchers specialize or have basic skills in writing surveys and
analyzing survey data. But in the last few years, many more tools have become
readily available to us.
We recently conducted a quick survey (there's the first
bias!) asking people about their opinions to a number of different common
research methods ('What is your opinion about participating in these kinds of
market research studies? Love, Like, Neutral, Dislike, Hate').
bias!) asking people about their opinions to a number of different common
research methods ('What is your opinion about participating in these kinds of
market research studies? Love, Like, Neutral, Dislike, Hate').
Obviously,
surveys were the favored method with 80% of more than 370 responders giving a
top 2 box response. Far behind surveys were communities at 49%, an interesting
result given that communities are a relatively new phenomenon. In third place at
44% was online focus groups. And in fourth place, mobile surveys received only a
32% approval rating, likely the result of our reliance on ancient survey designs
and our inability to keep up with the advanced features of smartphones. In last
place, face to face research generated approval from only 23% of people.
surveys were the favored method with 80% of more than 370 responders giving a
top 2 box response. Far behind surveys were communities at 49%, an interesting
result given that communities are a relatively new phenomenon. In third place at
44% was online focus groups. And in fourth place, mobile surveys received only a
32% approval rating, likely the result of our reliance on ancient survey designs
and our inability to keep up with the advanced features of smartphones. In last
place, face to face research generated approval from only 23% of people.
One of the main things we worry about as researchers is how
satisfied people are with the research process. We know that as satisfaction
declines, so do response rates. Well, one key feature of satisfaction is ensuring
that we allow people to participate in research using a method they like.
satisfied people are with the research process. We know that as satisfaction
declines, so do response rates. Well, one key feature of satisfaction is ensuring
that we allow people to participate in research using a method they like.
Let's consider 18 to 24 year olds, the most coveted, fawned
after group of people that researchers want to listen to. Though 80% of people
approved of surveys, only 75% of young people did. What they wanted more than
any other age group was mobile surveys (56% vs 32%). We'd better step up our
mobile survey skills and technology!
after group of people that researchers want to listen to. Though 80% of people
approved of surveys, only 75% of young people did. What they wanted more than
any other age group was mobile surveys (56% vs 32%). We'd better step up our
mobile survey skills and technology!
What about people who have a lot of education, another
highly sought after group? While 73% of them approve of surveys, they are also more
interested in focus groups than average (39% vs 31%). Granted, getting people
to attend a focus group may be difficult but there is an important group of
people who like them and should be offered them.
highly sought after group? While 73% of them approve of surveys, they are also more
interested in focus groups than average (39% vs 31%). Granted, getting people
to attend a focus group may be difficult but there is an important group of
people who like them and should be offered them.
Try an online focus group (51% vs 44%) if you'd like to interest
full-time employed people. Interested in higher-income people? Forget surveys
and go straight to communities (57% vs 49%). Need to listen to people from
different ethnicities? Try mobile surveys to interest African American people (41%
vs 32%), communities to interest Asian people (62% vs 49%), or focus groups to interest
Hispanic people (31% vs 44%).
full-time employed people. Interested in higher-income people? Forget surveys
and go straight to communities (57% vs 49%). Need to listen to people from
different ethnicities? Try mobile surveys to interest African American people (41%
vs 32%), communities to interest Asian people (62% vs 49%), or focus groups to interest
Hispanic people (31% vs 44%).
So who hates face to face research more than researchers? With
just 13% approval, don't even think about conducting face to face research with
women aged 18 to 24.
just 13% approval, don't even think about conducting face to face research with
women aged 18 to 24.
Annie Pettit, PhD is the Chief Research Officer at Peanut Labs, a company specializing in self-serve panel sample. Annie is a methodologist focused on data quality, listening research, and survey methods. She won Best Methodological Paper at Esomar 2013, and the 2011 AMA David K. Hardin Award. Annie tweets at @LoveStats and can be reached at annie@peanutlabs.com.