Guest Post: How much is enough when it comes to Voice of Customer?

jl1Jonathan Levitt has spent the last 5 years as a pioneer in the voice of customer analytics space. Through his speaking, writing, and evangelism, he was instrumental in legitimizing voice of customer analytics at a time when traditional web analytics still dominated the online business intelligence conversation. Jonathan has worked with world leading brands like Bank of America, Verizon, Dell, Procter & Gamble, Ford, and Reebok and has been featured in several industry publications including 1to1 Magazine, ClickZ, DM News, and MediaPost.

One of the best sources of business intelligence for companies of any size is raw Voice of Customer data.

This is particularly true for start-ups, where early, frequent, and consistent interaction with customers is critical to getting off the ground. The more customer-centric your decision making processes are from day one, the more likely you will get to the next stage in the development and maturation of your business plan.

This explains the recent growth in the selection of free and low cost Voice of Customer collection tools. User Voice, Kampyle, Survey.io, 4Q Survey (disclaimer: I helped conceive and build 4Q) — all of these are examples of popular Voice of Customer collection tools that can provide site owners with a pipeline of cheap and actionable visitor-sourced insights.

Once you put on the VoC practitioner’s hat, however, questions about respondent count size inevitably come up. Simply put, you need a way of knowing how much data is enough.

At what point can you act on the findings coming through your shiny new tools, with full confidence that you have collected a representative sample of your audience? If you’ve been running a User Voice customer feedback tool for 3 weeks and you’ve only collected 20 respondents, is that enough to act on? These are certainly agonizing questions for a data-centric marketer.

Now’s the time to start glancing over enviously at the big sites, because they don’t have this problem. The laws of probability are such that feedback from 500 respondents is usually enough to deliver reliable data at even the strictest confidence intervals. A big site like Dell.com can pull in 500 respondents within a day or two; at that clip, statistical significance comes through in a heartbeat.

But since your traffic generation muscle isn’t likely to match Dell.com’s anytime soon, I’ll let you in on a little secret: for small, startup websites that want immediate answers to their questions, the size of your sample almost doesn’t matter.

Here’s why. Representative feedback sampling requires a known population that is relatively stable and doesn’t fluctuate all that much–basically, a predictable population that will yield reproducible results. But the visitor bases of small, startup websites are anything but stable, especially if the websites are in a voracious traffic acquisition mode. The reality is that the composition of their online audiences is constantly shifting, which seriously undermines any effort at scientific VoC measurability.

This is one case where directional data can be just as powerful as representative data. I’m not saying you should blow up your website and start from scratch because of 1 piece of negative feedback, but you don’t need more than 20-25 pieces of feedback to really get started. So, forget about the science and the stats, and focus instead on segments of visitor discontent.

Look for repetitions and common patterns in your feedback; group similar items together and focus on sectional site optimization . If you notice two or three pieces of feedback that are eerily similar, then chances are your respondents are surfacing a real issue that’s resonating far deeper in your growing visitor base.

Voice of customer research can be a wonderfully responsive early warning system for a small website owner. Don’t get caught up in obsessing over respondent counts. If you’ve got 25 or so pieces of real visitor feedback at hand, you can go a long way in constructing a visitor-centric website experience that will help your website to grow and flourish.

Jonathan Levitt

Blunt – The Conversation Agency

  • chriswalker1

    Jonathan, absolutely agree….b2b tack that works extremely well yet is often neglected is applying voice of customer analysis to immediately improving sales. we interview hundreds of customers by phone and find that sampling of 5-10 is generally all that's needed to glean powerful feedback that can be used to dramatically improve quality of customer conversations and therefore sales….interested in your thoughts on customervoiceselling.com

    Chris

  • Anonymous

    Jonathan, absolutely agree….b2b tack that works extremely well yet is often neglected is applying voice of customer analysis to immediately improving sales. we interview hundreds of customers by phone and find that sampling of 5-10 is generally all that’s needed to glean powerful feedback that can be used to dramatically improve quality of customer conversations and therefore sales….interested in your thoughts on customervoiceselling.com

    Chris

  • http://ginevra.typepad.com ginevra

    [this is good] well said, Jonathan!

  • http://mizginevra.tumblr.com ginevra

    [this is good] well said, Jonathan!

  • jacqueswarren

    Qualitative Methodology has long been in use in social sciences where statistical sampling is not always available, or even relevent, such as in Life Story empirical data cases. Although it definitely poses epistemological challenges, one can draw conclusions from what is basically a sample of one. A whole lot of Anthropology is based on that (the particularities of a small population can still help you draw conclusions on human facts at a larger scale). It is all a matter of control, and understanding what can be said with what level of certainty. But again, a very small sample can still be very insight rich.

  • Anonymous

    Qualitative Methodology has long been in use in social sciences where statistical sampling is not always available, or even relevent, such as in Life Story empirical data cases. Although it definitely poses epistemological challenges, one can draw conclusions from what is basically a sample of one. A whole lot of Anthropology is based on that (the particularities of a small population can still help you draw conclusions on human facts at a larger scale). It is all a matter of control, and understanding what can be said with what level of certainty. But again, a very small sample can still be very insight rich.

  • jacqueswarren

    Qualitative Methodology has long been in use in social sciences where statistical sampling is not always available, or even relevent, such as in Life Story empirical data cases. Although it definitely poses epistemological challenges, one can draw conclusions from what is basically a sample of one. A whole lot of Anthropology is based on that (the particularities of a small population can still help you draw conclusions on human facts at a larger scale). It is all a matter of control, and understanding what can be said with what level of certainty. But again, a very small sample can still be very insight rich.

  • Pingback: How much is enough when it comes to Voice of Customer? | Blunt - The Conversation Agency

  • Anonymous

    Così Nike si fa a sapere se la tua prospettiva è una misura con quello che hai da offrire? è necessario chiedere loro questa domanda verso la fine della vostra discussione problem solving sta risolvendo il problema una priorità assoluta o qualcosa che è in secondo piano per ora?