Jonathan Levitt has spent the last 5 years as a pioneer in the voice of customer analytics space. Through his speaking, writing, and evangelism, he was instrumental in legitimizing voice of customer analytics at a time when traditional web analytics still dominated the online business intelligence conversation. Jonathan has worked with world leading brands like Bank of America, Verizon, Dell, Procter & Gamble, Ford, and Reebok and has been featured in several industry publications including 1to1 Magazine, ClickZ, DM News, and MediaPost.

One of the best sources of business intelligence for companies of any size is raw Voice of Customer data.

This is particularly true for start-ups, where early, frequent, and consistent interaction with customers is critical to getting off the ground. The more customer-centric your decision making processes are from day one, the more likely you will get to the next stage in the development and maturation of your business plan.

This explains the recent growth in the selection of free and low cost Voice of Customer collection tools. User Voice, Kampyle, Survey.io, 4Q Survey (disclaimer: I helped conceive and build 4Q) — all of these are examples of popular Voice of Customer collection tools that can provide site owners with a pipeline of cheap and actionable visitor-sourced insights.

Once you put on the VoC practitioner’s hat, however, questions about respondent count size inevitably come up. Simply put, you need a way of knowing how much data is enough.

At what point can you act on the findings coming through your shiny new tools, with full confidence that you have collected a representative sample of your audience? If you’ve been running a User Voice customer feedback tool for 3 weeks and you’ve only collected 20 respondents, is that enough to act on? These are certainly agonizing questions for a data-centric marketer.

Now’s the time to start glancing over enviously at the big sites, because they don’t have this problem. The laws of probability are such that feedback from 500 respondents is usually enough to deliver reliable data at even the strictest confidence intervals. A big site like Dell.com can pull in 500 respondents within a day or two; at that clip, statistical significance comes through in a heartbeat.

But since your traffic generation muscle isn’t likely to match Dell.com’s anytime soon, I’ll let you in on a little secret: for small, startup websites that want immediate answers to their questions, the size of your sample almost doesn’t matter.

Here’s why. Representative feedback sampling requires a known population that is relatively stable and doesn’t fluctuate all that much–basically, a predictable population that will yield reproducible results. But the visitor bases of small, startup websites are anything but stable, especially if the websites are in a voracious traffic acquisition mode. The reality is that the composition of their online audiences is constantly shifting, which seriously undermines any effort at scientific VoC measurability.

This is one case where directional data can be just as powerful as representative data. I’m not saying you should blow up your website and start from scratch because of 1 piece of negative feedback, but you don’t need more than 20-25 pieces of feedback to really get started. So, forget about the science and the stats, and focus instead on segments of visitor discontent.

Look for repetitions and common patterns in your feedback; group similar items together and focus on sectional site optimization . If you notice two or three pieces of feedback that are eerily similar, then chances are your respondents are surfacing a real issue that’s resonating far deeper in your growing visitor base.

Voice of customer research can be a wonderfully responsive early warning system for a small website owner. Don’t get caught up in obsessing over respondent counts. If you’ve got 25 or so pieces of real visitor feedback at hand, you can go a long way in constructing a visitor-centric website experience that will help your website to grow and flourish.

Jonathan Levitt

Blunt – The Conversation Agency

Related posts:

1. brilliant example of surveying users about a feature before it is built
2. Guest Post: How I increased traffic 1,176% in 24 hours

I have a problem with web analytics.

The whole notion of a web visit as a rigid set of steps that users follow is incompatible with how we use the web today. Visitors browse around the site, taking their time, exploring and interacting. Occasionally, they complete some kind of action we want—inviting their friends, buying something, and so on.

For a couple of years, I’ve been thinking about web visits in terms of two fundamental building blocks: Places and tasks. If you look at your site as a series of places and tasks, you’ll think differently about how and what you should be watching.

Places: Where users hang out

A place is somewhere on the site that a user hangs out. On Reddit, this might be where they’re reading submissions. There are small actions they can take—opening linked stories in new tags, or voting things up and down. On Wikipedia, they might be reading a story. In Google Apps, perhaps they’re working on a spreadsheet.

When a user’s in a place, we care about their productivity. Are they able to vote up or down smoothly? Can they read the story quickly, and do images load? Are they successfully building a business projection in that spreadsheet? We also care about disengagement—a user who gets bored of reading articles and goes elsewhere, for example.

Tasks: When users have a mission

By contrast, a task is something the user sets out to accomplish. It’s several steps, and some of those steps don’t happen on the website itself. And a task puts the user in a different mode of operation. It’s the Reddit user creating an account for themselves, or submitting a new link. It’s the Wikipedia reader deciding to edit a page. Or it’s the Google Apps user sharing their spreadsheet with someone.

When a user’s trying to accomplish a task, we care about their accomplishment of the task. Did the invite result in a new enrolment? Did they complete the purchase? Was the edit of the article ultimately saved? Were they able to add the widget to their dashboard?

A new way to look at sites

Looking at websites as collections of places and tasks quickly underscores the limitations of page-centric, funnel-minded web analytics.

For one thing, you quickly realize that you need to instrument places and tasks very differently: Places need analysis of actions within the place (How many videos did he watch? How often did he pause them? Did he see the ad?) while tasks need analysis of progress (Did he send the mail? Did it bounce? Did the recipient act on it?)

What’s more, traditional page-centric instrumentation won’t work. Actions happen at the sub-page level, with components of a page; often, this involves a Rich Internet Application like Flash or Silverlight.

To further complicate matters, tasks often involve steps beyond the view of analytics, such as e-mail invitations, instant messages, RSS feeds, and third-party sites. Tracking the accomplishment of a task across multiple systems is a challenge, with all manner of tracking cookies, dynamic URLs, and embedded GIFs used to try and follow the task to completion.

Paving the way to places and tasks

There’s lots of good innovation going on in this realm. Steve Souders’ Episodes model creates a way for designers to instrument user actions cleanly even within a page, which is ideal for “places” analytics. Google recently unveiled the ability to send back events (like upvoting a story) within a page.

At the same time, Kissmetrics’ Product Planner, which showcases design workflows, is all about tasks. In some cases the tasks are linear—buy something—while in others, they’re part of a feedback loop such as inviting others that leads to viral adoption.

What can you do to get started?

Most web operators have a mental map of their site. Some even draw it on a wall. You can map out a site, consisting of places and tasks, in this way.

For each place, make a note of all the events you care about. Including timing events (“a video starts playing”) and user interactions (“user upvotes and the button’s color changes.”) Also identify the actions that initiate a task (such as “share this spreadsheet.”)

For each task, make a note of all the steps you want to track, including those that aren’t on your site. Identify the key metrics you should know (for a mail send, for example, this might be bounce rate, open rate, and click rate.)

Then the next time you’re presenting your web monitoring results, overlay them on the map. For each place or task, show the analytics (what the users did) and the user experience (whether they could do it.) If you have psychographic information (why they did it) such as surveys, or usability metrics (how did they do it), include that as well.

The end result is a much more accurate representation of the ebb and flow of your online business. It will probably reveal significant gaps in your web visibility strategy, too—but at least now you’ll know where your blind spots are.

No related posts.

I’m currently in the middle of writing the Web Analytics chapter for the book, and my gosh – things have changed so much in fifteen years.  This screenshot is from the program “GetStats”, one of the first web analysis tools to exist.  I ran it using watchingwebsites.com logs (I had to parse them through sed & awk to change their log format to CLF for it to work).  Notice how it took 7 and a half minutes to process as many lines!

I was talking to the author, Kevin Hughes about GetStats and the state of web analytics when he first wrote it.  “Actually getstats wasn’t the first Web server log analysis tool, but it was very influential in terms of the way the data was presented and summarized.  Roy Fielding with wwwstat was the first as far as I can recall to present statistics in an easy-to-read paragraph summary form, that I think was written in Perl.  I also took ideas from Thomas Boutell (wusage) and Eric Katz (WebReport).”

Web analytics tools began by telling us how many hits we had on the site, but that doesn’t do much today to tell us what’s really happening with our sites.  The tools went through many evolutions before they got to where we are today – simple metrics, a few KPIs and actionable information.  I’ll touch a bit on this in the book; we’ll also cover implementation methods, advantages, limitations and deployment impact of web analytics tools.

The book is days away from having a completed 1st draft.  I can’t wait to send the complete manuscript out to the reviewers!

Related posts:

1. Google Analytics Alerts: the start of a complete view?