We’re in Amsterdam this week, presenting at a Measureworks conference on web performance and optimization and attending a Tweetup.

Our host, Jeroen, told us yesterday that since the introduction of GPSes in Amsterdam, traffic accidents in the narrow-streeted city have risen significantly. Many people are focused on their instruments, rather than looking around them. This made me think of some issues I’d seen with web advertising recently that would have been hard to detect through instruments alone, and underscored some of the shortcomings of a purely instrument-driven analytics approach.

With much fanfare, Microsoft launched Windows 7. By many accounts, it’s a good operating system, despite the widely derided launch parties they tried to encourage (which, to be fair, did get people talking about the launch.) The launch involved a massive online ad buy, as well as a new online store for the company. Two aspects of this launch caught my attention: The differences between regional stores, and the state of video advertising.

The differences between regional stores

Here’s how Microsoft is selling its wares in the US. There are five steps from landing page to checkout.

Note that you can click on the cropped images here to see a larger view of the page from which it came, if you want to better understand it in context.

First, there’s an invitation to get Windows 7 on the home page. Decent, and since I want it, I’ll click it.

Okay, I’m given some options, but the one I want most is to buy the thing. The other options are stuff like a store locator, but since this is 2009 and I have an Internet connection, I’m most interested in buying it now.

I get to choose which one I want — there are three versions. I can see pricing. Great, I’ll click “Buy now.”

I can choose some options (full or upgrade version, download or shipped version.) The defaults are okay, so I’ll click “Add to Cart”.

The cart contents are confirmed, and I can check out. I have some bonus stuff — a chance to second-guess myself on not getting the backup, which increases order cart size; a chance to buy more than one; and a chance to tell someone else about the purchase. This is good e-commerce behavior. But I’m just going to buy one for now.

That was what I expected, and I have an order, right? Well, not if you’re in Canada. Let’s try the same thing on the Canadian Microsoft site.

And now, Canada:

I start by clicking “Get it now” on the home page. This isn’t exactly the same as the image on the US site, but it’s the same text.

Okay, I’ve got the same three versions from which to choose.

The price is higher, which is too bad because the Canadian dollar is roughly 1:1 with the US dollar right now, so this is a 20% “not paying attention to International currency rates” tax. But that’s nothing new on the Internet.

I also notice that the “buy now” button is called “shop now” in Canada. I assume that’s just the result of A/B testing and that Canadians prefer “shop” to “buy” for some reason. So I click it.

Here’s what I get. Wait, what?

Instead of confirming options and putting it in my cart, I see a screen showing me the names of stores.

Let’s be clear: Store information is not what I wanted. There’s nothing on this page saying, “we’re sorry, in Canada we don’t sell products online; you’ll have to go to a store.” Forget for a minute the fact that Canadians can’t buy Windows software online — astonishing as that may be. This is a completely jarring workflow. After what felt like a normal shopping cart pattern, I was suddenly given a page that made no sense. When this actually happened, I tried the workflow three times before realizing what had transpired.

If you’re a web analyst at Microsoft, would you see this? Likely not — you’d notice a large number of exits from this page, and assume you’d done your job of informing visitors where they could pick up a copy of your new product. Instead, you are experiencing huge abandonment.

Relying on your instruments too much

As the Canadian Microsoft store example shows, sometimes your instruments won’t show you something plain and obvious; you have to use common sense. A simple message on the final page saying, “we don’t sell software online in Canada” would alleviate the misunderstanding and make the page successful. Microsoft could even have a link explaining why — and track traffic to that link, to gauge Canadian interest in online purchases, or solicit comments that might mitigate an outcry from Canadians trying to convince Redmond they’re ready to buy products over the web.

Let’s talk about video

Another part of the advertising blitz around Windows 7 is the videos they’re embedding into CTV’s player website. This is a site on which Canadians can watch recently-aired programs (similar to Hulu in the US and the BBC iPlayer in the UK.)

The CTV website isn’t very sophisticated. The most egregious problem with the site is that it can’t interleave commercials and content cleanly. Where Hulu switches smoothly between a program and an ad, the CTV site actually pops out of full-screen mode each time there’s a commercial, requiring the viewer to re-maximize the player.

Just how bad is this inability to handle interactions with online video? When Microsoft advertises Windows 7 on the site, they run a static image for thirty seconds inviting the viewer to click on an ad at right.

That’s completely broken. It underscores just how nascent online video advertising is today. The only way that CTV can let Microsoft segment its respondents is to force them to pop out of full-screen, then invite them to click on a static image which can be imagemapped.

To do video advertising properly, advertisers will soon demand:

• A referring tag showing where on the screen the visitor clicked (in case there are several visual components)
• The timecode within the ad (in case the ad covers several features.)
• The programming information (what show was it?) and other facts that can be used for demographic targeting

This stuff is available in more advanced analytics and video delivery platforms. But the fact that Microsoft had to waste 30-second video clips on a static image shows just how far we still have to go before video advertising delivers on its promise.

No related posts.

Disclaimer 1: All site-related data found in this post comes from compete.com.  The company was kind enough to give us a “pro account” to help us research the O’Reilly book that we wrote called Complete Web Monitoring (thanks, you rock!).  However, compete.com did not sponsor this post (nor did any company, for that matter).  And yes, we know – compete.com numbers are simply estimates.

Disclaimer 2: I (Sean) worked for Akoha as Community Gardener while we launched at TechCrunch50 2008; but I’m now doing metrics, web analytics, performance, and social computing consulting.  The views found below are mine, and do not reflect those of Akoha in any way.  For the record, Akoha is awesome!

About us: This post was written by Sean Power with Alistair Croll.

Dear TechCrunch50 Startups,

Congratulations. You made the list. You’re finally launching, and that pent-up frustration of not being able to tell people about it for a month is almost at an end. Now, you have to live with a weekend of cold, hard fear that your demo will explode. You’ve got an interesting week ahead, and I know you’re short on sleep, so let me get to the point quickly.

You’re probably excited about the TC50 bump. I first saw the term used by Josh Kopelman of First Round Capital on the RedEye VC blog. The bump refers to the pounding your website is about to experience from TC50 attendees, readers, bloggers and their friends.  It’s not to be underestimated.  Here’s a glimpse at how the bump looked like for all TC50 startups in 2008.  If you squint a little, you’ll see Akoha somewhere in there!:

This is an unprecedented influx of attention. It may be the single biggest traffic spike you’ll ever experience. Thousands of visitors will drive by your site, stay for a minute, and leave — never to return. After the bump, you’ll feel a tremendous rush of adrenaline, then deep, soul-sucking disillusionment as your traffic dwindles back to its former levels.

Don’t waste this opportunity. If you take the right steps, you can make the most of your fifteen minutes of fame.

Being Talked About Is Nothing. Being Remembered Is Everything,

You’re probably celebrating the traffic you’ll get to your site. But you shouldn’t be, unless you capture the minds and hearts of your visitors.  The best way to do that is to target them.

On a normal day, you know very little about your site traffic. But today, you know lots. And that means you can tailor the experience to your audience. Let’s say you knew for a fact that 95% of your visitors loved the color red.  You’d probably redesign your site to make it bright red, right?  Well, next week, over 85% of your visitors will all come to you through TechCrunch somehow. Think about it. For a single week, you know exactly what kind of visitor will be visiting you.

TechCrunch50 visitors will load your site, scan quickly, and may even register (if registration isn’t broken, and is simple enough). Then they’ll leave.  In that crucial 10 seconds of play time, your job is to make sure that as many people remember your site as possible, and that you gain the ability to reconnect with them later, and to engage with them.

During TC50 2008, Robert Scoble blogged about Demo and TC50 websites as “sucking big time”. Knowing that he was an important influencer, we put his face on the front page of Akoha, with a speech bubble saying “If your name is Robert Scoble, click here!”, which brought the user to a small blog post explaining where we were going with our site design.

It worked. Scoble reciprocated in kind:

Being memorable is everything at an event like TechCrunch 50.  Find ways to be remembered by those who visit you.

Gathering Data

Since you’ve just launched, now is the time to start figuring out how unsolicited users react to your site. You’ve got a great lab for a few short, sweet days. Try a couple of site designs, and see which one works best.  This is where simple abtests becomes useful. Make sure you target people specifically. And turn on all the monitoring you can (without making the site slower.) It’s an excellent time to make baselines.

Google Analytics Is Not Enough

It really isn’t. For one thing, it doesn’t show you results fast enough in a flash crowd situation. Consider a more real-time analytics tool like Getclicky or Woopra, so you can see traffic as it happens — not the day after. Then find someone to watch it. While you’re on stage, or schmoozing investors, have someone back at headquarters (or whatever you call your mom’s basement) looking at the traffic.

But even if you have up-to-the-minute analytics, it’s not enough. It’s one thing to know what your users did, but chances are that you don’t even have goals set up in Google Analytics, and even if you do, you’re only measuring if people reached the goals, not why.

Measure Peoples’ Behaviors

Take a look at your site right now.  In two years, it will look completely different, because you’ll learn. What if you could speed up that clock? You can, if you measure what people do on your site and iterate rapidly.

A Web Interation Analytics tool measures where people click (or don’t), helping you figure out where your design flaws are without even asking people.  Go install CrazyEgg or ClickTale right away.  You can’t afford to miss out on this data. If you’re really serious, put Coradiant or Tealeaf in front of your site.

Ask People What They Think

People are weird. They’ll do all kinds of things you don’t expect. It’s one thing to watch what they do — but often, the real key to unlocking your business potential is to know why they did it. The simple way to find this out is to ask them. Do yourself a favor and set this up now.  Install 4Q, Kampyle, GetSatisfaction or UserVoice right away on your site.  Have the guts to ask what people like and don’t like about your site so you can fix it faster than your competitors.

And don’t just rely on tools. Be on Twitter, and be reachable. When that girl/guy in your basement isn’t watching analytics, have him or her respond to people online. When Alistair & I present at conferences, we take turns teaching and reading Twitter — fielding questions from the audience, responding to folks who couldn’t be there, and seeing if the folks in the back of the room can hear us. Do the same thing for your site.

Make Sure Your Site Can Weather The Storm

If your site is slow, people will leave.

This isn’t just a platitude: Google and Microsoft have both released empirical evidence that latency correlates with departure, and reduces conversion rates. Worse, people who had a slow experience will use the site less even when it gets faster again.

Bounce rate (the number of people that see one page and leave) may be a function of site latency. So test it! Call Keynote, Gomez, AlertSite or WebMetrics and start testing your site’s speed and availability every 5-15 minutes. Still haven’t got funding? Consider AlertFox or pingdom.  Are you at TC50, trying to load your site and it’s not working?  Not sure if it’s your computer, TC50’s Internet or your site?  Use AOL Page Test to test it remotely and find out why it’s sluggish.

Don’t forget to test the key pages and functions you want people to use, such as enrollment.

Have a Simple Call To Action

You’ll have a lot of folks on your site. Make it painfully obvious what you want them to do, whether that’s signing up, giving you permission to contact them later, trying it out, or telling friends. Don’t give them several choices — give them one. If 95% of people are just there to find out what you do, tell them. You can show them press releases or a list of your board members later. And consider tightening up your site copy, too. Less words is good.  Still not sure?  Talk to Josh Porter.  He can help.

Listen and Learn

Ultimately, you want to hear what people are saying about you elsewhere — not just on your site — and respond to the criticism and compliments.  Call ScoutLabs, TechRigy or Radian6 right away.  In most cases, all you need is a credit card and off you go.  Set up searches for keywords that matter and find conversations that concern you.  Twitter search is your friend. Google Alerts are too. There’s no better way to make a first impression than by actually being there to make a first impression.

Running The Booth?

Read this excellent post by Jason Calacanis on the subject.  It will help you avoid disillusionment when you come back from the launch party.

Learn From The Jedi Masters

Once TechCrunch50 is over, you’ll have very little time to rest on your laurels.  Chances are that your startup will die in the next year.

(Don’t die!  We don’t want you to die!)

Learn with the masters instead.  Follow Eric Ries, Dave McClure and Avinash Kaushik religiously.

Come And See Us In November (Or Call Us Before)

It’s awesome that you’ve come this far. Hopefully something in this list will help you learn from the storm, and turn the bump into a ramp. Whatever the case, once the dust settles you’ll have a lot of data to dig through. We’d love to help.

If you’re a finalist at TechCrunch50, we’re impressed that you’ve read all the way up to here.  Thanks!  We know you have so many things to do this weekend, and we appreciate that you’ve given us a few minutes of your time.

We feel for startups, we love ‘em!  So we’d like to make you an offer.  Once the next couple of weeks are done, give us a call.  We’ll spend an hour with you on the phone or online, gratis, and see if we can help you sort out your data, no strings attached.  You can reach Alistair and myself on Twitter, or simply email me directly – sean at httpd dot org.

We’ll also be doing an eight hour bootcamp called “Communilytics” at Web2Expo in New York in November on the subject of community metrics; if you’re coming, let us know.

Good luck out there, and knock ‘em dead!

- Alistair & Sean

ps; For those of you curious to know what sort of traffic the TechCrunch 08 attendees received, you’ll find the result of our findings below.

The Bump By The Numbers

When analyzing the TC50 08 finalist sites, the numbers from compete.com can be misleading.  For most sites, the traffic volume is relatively low, and therefore more prone to inaccuracies due to small sample size.  Consider that a disclaimer.  Averages were calculated between the months of October 08 and August 09 in order to avoid skewing the data as a result of the TechCrunch Bump.

It would be useless to lump all the sites in one bucket.  Grouping transaction and collaboration sites side by side is like admiring the similarities of hippopotamuses and cream puffs.

To get better insight, we must segment into the four types of sites found on the Internet.

Executive Summary

• Media sites showed relatively large traffic volumes, with unique visitor count often going up and to the right.
• Collaboration sites showed consistent traffic patterns from the month of October to august, but had no month to month average growth.
• The TechCrunch bump for transaction sites represented a small portion (8%) of the total unique visitor count they would receive for the next 11 months.
• The TechCrunch bump for SaaS portal on average represented roughly the total amount of traffic they would receive for the next 11 months.

Media Sites

These sites offer content that attracts and retains an audience. They make money from that content through sponsorship, advertising, or affiliate referrals. Search engines, AdWords-backed sites, newspapers, and well-known bloggers are media properties.

Between the months of OCT 08 (one month after TechCrunch50 ‘08) and August 09 (one month before TechCrunch50 ‘09), media site finalists:

• received, on average, 1,340,000 unique visitors (860,000 unique visitors with outliers removed (MIN and MAX)).
• showed peaks with a 41% increase in unique visitors, a positive influx of 148,000 unique visitors for that month.
• showed valleys with a %29 decrease in unique visitors, a loss of 171,700 unique visitors for that month. (ouch!)
• with the highest unique visitor count (MAX) showed an average monthly growth of 21.33% from 84,600 unique visitors totaling 3,530,000 unique visitors.
  
• with the lowest unique visitor count (MIN) showed an average monthly growth of 2.57% from 16,800 unique visitors totaling 99,600 unique visitors.

On average, the TechCrunch bump brought 57,900 unique visitors (55,300 unique visitors with outliers removed) for the month of September 08 and represented 8.55% of the total traffic they would receive in the following 11 months (8.17% with outliers removed).

Transaction Sites

A site that wants visitors to complete a transaction—normally a purchase—is transactional. There’s an “ideal path” through the site that its designers intended, resulting in a goal or outcome of some kind. The goal isn’t always a purchase; it can also be enrollment (signing up for email) or lead generation (asking salespeople to contact them), and that goal can be achieved either online or off.

Between the months of OCT 08 (one month after TechCrunch50 ‘08) and August 09 (one month before TechCrunch50 ‘09), transaction site finalists:

• received, on average, 272,000 unique visitors (124,000 unique visitors with outliers removed (MIN and MAX)).
• showed peaks with a 700% increase in unique visitors, a positive influx of 65,700 unique visitors for that month.
• showed valleys with a %49 decrease in unique visitors, a loss of 23,700 unique visitors for that month. (ouch!)
• with the highest unique visitor count (MAX) showed an average monthly growth of 15.49% from 113,800 unique visitors totaling 2,916,000 unique visitors.
  
• with the lowest unique visitor count (MIN) showed 0 growth and traffic.  (oof).

On average, the TechCrunch bump brought 35,300 unique visitors (26,600 unique visitors with outliers removed) for the month of September 08 and represented 8.83% of the total traffic they would receive in the following 11 months (6.64% with outliers removed).

Collaboration Sites

On these sites, visitors generate the content themselves. Wikis, news aggregators, user groups, classified ad listings, and other web properties in which the value of the site is largely derived from things created by others are all collaborative.

Between the months of OCT 08 (one month after TechCrunch50 ‘08) and August 09 (one month before TechCrunch50 ‘09), collaboration site finalists:

• received, on average, 138,000 unique visitors (86,400 unique visitors with outliers removed (MIN and MAX)).
• showed peaks with a 316% increase in unique visitors, a positive influx of 29,500 unique visitors for that month.
• showed valleys with a %63 decrease in unique visitors, a loss of 24,600 unique visitors for that month. (ouch!)
• with the highest unique visitor count (MAX) showed an average monthly growth of -1.02% from 108,000 unique visitors totaling 997,000 unique visitors.
  
• with the lowest unique visitor count (MIN) showed an average monthly growth of -50.70% from 1,400 unique visitors totaling 5,000 unique visitors.

On average, the TechCrunch bump brought 37,600 unique visitors (26,500 unique visitors with outliers removed) for the month of September 08 and represented 47.23% of the total traffic they would receive in the following 11 months (29.58% with outliers removed).

Software-as-a-Service Sites

These sites are hosted versions of software someone might buy. SaaS subscribers expect reliability and may pay a monthly per-seat fee for employees to use the service. Revenues come from subscriptions, and a single subscriber may have many user accounts. On some SaaS sites, users are logged in for hours every day.

Between the months of OCT 08 (one month after TechCrunch50 ‘08) and August 09 (one month before TechCrunch50 ‘09), software-as-a-service site finalists:

• received, on average, 62,000 unique visitors (47,000 unique visitors with outliers removed (MIN and MAX)).
• showed peaks with a 337% increase in unique visitors, a positive influx of 18,200 unique visitors for that month.
• showed valleys with a %65 decrease in unique visitors, a loss of 15,400 unique visitors for that month. (ouch!)
• with the highest unique visitor count (MAX) showed an average monthly growth of 1.52% from 26,000 unique visitors totaling 250,000 unique visitors.
  
• with the lowest unique visitor count (MIN) showed an average monthly growth of 64.46% from 1,400 unique visitors totaling 5,500 unique visitors.

On average, the TechCrunch bump brought 23,800 unique visitors (21,800 unique visitors with outliers removed) for the month of September 08 and represented 100.23% (!!!) of the total traffic they would receive in the following 11 months (91.81% with outliers removed).

Notes:

Big thanks to wa7iut for helping me with the stats!

Alfabetic.net was not included because it was impossible to determine what kind of site alphabetic.net is (site is down, nothing on Google cache or archive.com and the ustream demo doesn’t make it clear).

We used a combination of fotopedia.com and fotonauts.com data due to a site rebrand.

footnote.com was not included in the analysis.  By the time that they were presenting at TC50, they had 1.3million unique visitors a month, with 1.4 million unique visitors in December 2007.  Statistically (and from a TC50 perspective), footnote.com was too much of an anomaly.  We do, however, think that their site is totally cool.

Related posts:

1. DemoCamp Guelph

We’re doing a presentation that’s excerpted from the book at DemoCamp Guelph tonight. Should be an interesting conversation; we have an “exercise” planned. Sean can’t be here (he was at Podcamp and has to get real work done after a weekend of editing the 400+ figures in the text!) but will be joining on Twitter. If you have photos from the event, or questions for Sean, we’ll be using the #CWM hashtag (for Complete Web Monitoring, the title of the book.)

One of the projects we’ve been working on is trying to create a single, comprehensive overview of the Complete Web Monitoring process. Here’s where we’re at (and an early glimpse at a poster we’re working on.)

First of all, a complete monitoring strategy includes the many questions a web analyst needs to answer:

• Web analytics (“what did they do?”)
• Web Interaction Analytics (“how did they do it?”)
• Voice of the Customer (“why did they do it?”)
• Both synthetic and real user performance monitoring (“could they do it?”)
• Community monitoring (“what are they saying?”, “who’s talking?”, and “where are they saying it?”

Any strategy also has to look at several different stages in monitoring:

• Arrival (“I visited the site”)
• Usage (“I played with it”)
• Engagement (“I’m a part of it”)
• Revenue (“I paid for it”)
• Referrals (“I spread the word”)

If these look somewhat like Dave McClure’s Pirate Metrics, it’s because he’s awesome and we borrow heavily from his thinking on startup metrics. Anyway, this PDF is a work in progress of trying to align the big questions analysts need to answer with the various stages of visitor engagement. Once we sex it up a bit, we’ll make some posters.

I’ll put the DemoCamp slides up here shortly.

Related posts:

1. Slides from DemoCamp Guelph 08
2. An Open Letter To All TechCrunch50 2009 Startups: The TC “Bump”, What It Really Means & How To Navigate It