SAMPLE THIS: Only 0.25% of the population provides one-third of online responses

Throughout the past year, I have been sharing my concern about the accuracy of online research with my clients.  The problem; the dramatic shift to internet surveys has created a class of professional survey-takers that don’t always represent the opinions of the masses. 

Over $7 billion is spent each year on market research and a 2006 report by Inside Research projects that over one-third of survey spending now goes online.  It seemed to me to be inevitable that competition for consumers would emerge as researchers tried to build the massive panels required to capture the $1.3 billion flowing to online research.  My contention was that increasingly lucrative incentives might actually concentrate the audience and encourage some survey-takers to participate in hundreds of surveys a month, effectively becoming “professional respondents”.  Now, it seems, my hypothesis has been validated.   

In September, top research executives gathered in Chicago to discuss the challenges of market research.  Even stalwart marketers like Procter & Gamble Co., General Motors Corp., IBM and McDonald's shared their frustration over declining response rates, opinion fatigue, and samples that are not reflective of the population.  Research conducted by ComScore Networks was cited showing only 0.25% of the population provides 32% of responses to online surveys. 
[I hate to say, “I told you so!” so I’ll say, “I hope you listened.”]

To be clear, responses from a small sample of the population do not inherently mean that internet survey results are biased.   All research is representative.  However, my concern is overdependence on this single form of consumer insight.  A P&G concept test demonstrated this when online and mail surveys gave opposite results.  As Kim Dedeker, VP-consumer and market knowledge at P&G, put it, “"If I had only had the online result in this particular case, I would have taken a bad decision right to the top management." 

So what’s the solution?  It’s simple.  Talk to your customers as often as possible and take everything with a grain of salt.  I firmly believe that every customer contact is an opportunity to gain significant insight and contextualize traditional research.  Internet surveys, mail surveys, phone surveys, focus groups, intercept surveys and even in-home observation can all offer insight.  But don’t forget about powerful non-traditional approaches like having your executives call consumers every month, or having them spend a day at retail observing and talking to shoppers.  And, take it a step further by creating an internal blog for all employees to discuss their consumer learning.  The bottom line is that most consumer insight is valuable as long as you understand how it was gathered and the resulting implications on the data.   

Don’t just take my word for it, follow the “Expert Advice” recently published by AdAge:

“Challenge the research firm on how it manages survey panels and polices for 'professional respondents'... [and] If you plan to do it yourself, consider hiring a freelance consultant to raise the odds that your work will produce valid, actionable results.” 

Now that’s good advice! 

Related Links:

AdAge - online research advice and trends (pdf)

AdAge-Consumers Rebel Against Marketers' Endless Surveys

  Send Page To a Friend

^top
 

cmsolutions provides consulting and research-services focused on translating consumer insight into actionable strategic plans.  cmsolutions' research services give companies a unique understanding of their customers using our True Consumer Insights methodology.  This innovative approach to direct consumer research provides unrivaled clarity and depth of insight that will teach you what your REAL customers REALLY think. 

cmsolutions is also a leader in innovative approaches to Category Management, providing unparalleled results without cumbersome and ineffective processes.  Our 7-day Plan process can help your people develop a comprehensive and effective Category Business Plan in as little as seven work days with average sales increases of 14%!

cmsolutions clients range in size from small professional firms to Fortune 500 companies and span numerous industries including: consumer goods, retail, manufacturing, professional services, software/Internet, and non-profits. Our clients have created more than $200 million in value with our help, and we can help your company achieve similarly extraordinary results.  

cmsolutions was founded in 1998 by Bart E. Steiner, with the commitment that the company would only accept projects in which we can create at least $10 in value for every $1 a client invests.

And all of cmsolutions work is guaranteed.  
If you aren't happy with our work, we'll make it right, or refund your money. 
So lets get started building your business today!