Guest Post: How much is enough when it comes to Voice of Customer?

Jonathan Levitt — Tue, 03 Nov 2009 02:18:19 +0000

Jonathan Levitt has spent the last 5 years as a pioneer in the voice of customer analytics space. Through his speaking, writing, and evangelism, he was instrumental in legitimizing voice of customer analytics at a time when traditional web analytics still dominated the online business intelligence conversation. Jonathan has worked with world leading brands like Bank of America, Verizon, Dell, Procter & Gamble, Ford, and Reebok and has been featured in several industry publications including 1to1 Magazine, ClickZ, DM News, and MediaPost.

One of the best sources of business intelligence for companies of any size is raw Voice of Customer data.

This is particularly true for start-ups, where early, frequent, and consistent interaction with customers is critical to getting off the ground. The more customer-centric your decision making processes are from day one, the more likely you will get to the next stage in the development and maturation of your business plan.

This explains the recent growth in the selection of free and low cost Voice of Customer collection tools. User Voice, Kampyle, Survey.io, 4Q Survey (disclaimer: I helped conceive and build 4Q) — all of these are examples of popular Voice of Customer collection tools that can provide site owners with a pipeline of cheap and actionable visitor-sourced insights.

Once you put on the VoC practitioner’s hat, however, questions about respondent count size inevitably come up. Simply put, you need a way of knowing how much data is enough.

At what point can you act on the findings coming through your shiny new tools, with full confidence that you have collected a representative sample of your audience? If you’ve been running a User Voice customer feedback tool for 3 weeks and you’ve only collected 20 respondents, is that enough to act on? These are certainly agonizing questions for a data-centric marketer.

Now’s the time to start glancing over enviously at the big sites, because they don’t have this problem. The laws of probability are such that feedback from 500 respondents is usually enough to deliver reliable data at even the strictest confidence intervals. A big site like Dell.com can pull in 500 respondents within a day or two; at that clip, statistical significance comes through in a heartbeat.

But since your traffic generation muscle isn’t likely to match Dell.com’s anytime soon, I’ll let you in on a little secret: for small, startup websites that want immediate answers to their questions, the size of your sample almost doesn’t matter.

Here’s why. Representative feedback sampling requires a known population that is relatively stable and doesn’t fluctuate all that much–basically, a predictable population that will yield reproducible results. But the visitor bases of small, startup websites are anything but stable, especially if the websites are in a voracious traffic acquisition mode. The reality is that the composition of their online audiences is constantly shifting, which seriously undermines any effort at scientific VoC measurability.

This is one case where directional data can be just as powerful as representative data. I’m not saying you should blow up your website and start from scratch because of 1 piece of negative feedback, but you don’t need more than 20-25 pieces of feedback to really get started. So, forget about the science and the stats, and focus instead on segments of visitor discontent.

Look for repetitions and common patterns in your feedback; group similar items together and focus on sectional site optimization . If you notice two or three pieces of feedback that are eerily similar, then chances are your respondents are surfacing a real issue that’s resonating far deeper in your growing visitor base.

Voice of customer research can be a wonderfully responsive early warning system for a small website owner. Don’t get caught up in obsessing over respondent counts. If you’ve got 25 or so pieces of real visitor feedback at hand, you can go a long way in constructing a visitor-centric website experience that will help your website to grow and flourish.

Jonathan Levitt

Blunt – The Conversation Agency

The anatomy of support crowds

alistair — Tue, 05 May 2009 18:48:17 +0000

(I wrote up a detailed outline of this at Bitcurrent.)

I attended a panel on crowdsourcing support at the SIIA Software Summit. The panelists had some interesting statistics on what the crowds within an online support community are like, and what they look for.

Changing metrics for changing focus

On SAP Community Network, 90% of people consume information; 10% contribute it, and 1% are active. No news here — this is consistent with findings by Charlene Li, Jakob Nielsen, and others. But the data that mattered to the community changed as it matured:

Early on, SAP attracted people because it had content you couldn’t get anywhere else, so the metrics that mattered were those of a content publishing system — who’s creating content, what are people reading most, and how good is the content you’re creating.
After some time, the connections community established with other people started to matter more, and the focus shifted to tools for establishing connections — so analytics looked at who was befriending whom, regulating spam, and the like.
Eventually, the site was popular enough that a community existed in its own right, and it became a point system for ranking and thanks. The focus was a reputation management system — and the analytics had to track leaders, scoring, and so on.

This happened over a period of 6 years, and the company invested heavily in things like member recognition. Ultimately, community members with high rankings were able to use this on their LinkedIn profiles, because it’s a sign to potential employers of that person’s expertise and ability to work with others.

The goal of your community changes your magic 1%

Over at Lithium, they also have a 100:1 ratio of consumers to active contributors. But they point out that the nature of that 1% varies depending on the goals of the community.

If the goal of the community is to drive down costs, your ideal 1% is the folks who have the answers.
If it’s new product ideas you’re after, then you care about the 1% of members who ask the best questions.
If you’re trying to generate leads, your perfect 1% is the people who know others.

The payoff

The payoff for these communities is big. First of all, there’s the reduction in support costs. Each call that doesn’t happen saves the company $5-$10. But there’s also the fact that the community knows better than a single vendor. Every support problem has many moving parts — browser, router, carrier — and no one company knows all of the issues. But the community does. Vendors simply can’t afford to test with every possible combination. But communities, by definition, can.

Ultimately, support communities are one of the most popular, visible sources of community ROI. But expect to change the metrics you track as they mature and as the goals you’re after change.

No related posts.

Watching Websites » Who’s talking?

Guest Post: How much is enough when it comes to Voice of Customer?

The anatomy of support crowds

Changing metrics for changing focus

The goal of your community changes your magic 1%

The payoff