Alistair Croll & Sean Power blog on watchingwebsites.com about web technology, startups and analytics.  They are the co-authors of Complete Web Monitoring (O’Reilly, 2009) and contributors to Web Operations (O’Reilly, 2010).  If you like this post, you’ll probably like the one we wrote about analytics &  the TechCrunch bump.

Facebook recently introduced embedded commenting within websites, under the name Facebook Comments. Does this new model for commenting on posts help, or hurt, site engagement?  To find out, we compared two weeks’ worth of TechCrunch posts; 7 days before and 7 days after the site implemented the Facebook Comments feature.  Data geeks, you can find the source data here.

In order to reduce the outlier effect of posts with very high, or very small, levels of comment —the massively successful, or truly abysmal, ones—we trimmed the lowest and highest 5 percent of results. In this analysis, “All posts” refers to sums and averages on all posts published, whereas  ”average posts” refers to sums and averages on all posts with the 95th and 5th percentile values removed.  By trimming the results in this way, we hope to get a better representation of the effect of Facebook Comments on a typical post.

TL;DR

To be able to fully understand the scope of Facebook Comments, we’re missing a few critical factors which are only available to the owner of the site itself:

Your strong value proposition comes in two forms: your ability to drive users from your own platform to publishers, and your ability to prevent spam and trolling by forcing identity on all comments.  If you can claim the largest publishers, you have a chance at usurping Twitter’s position as the leading means of spreading awareness about a piece of news.

Those of us who are building web apps and services inevitably ask ourselves “if I build in the cloud, which one performs the best?”.  In this scenario, the word “best” is a misnomer – what we really mean is “what cloud can perform better given the particular needs of my app?”.  Well, Alistair set out to find those answers.

Along with Webmetrics, he published research that quantifies web application performance on Amazon, Google, Salesforce, Rackspace and Terremark.  Instead of using a “one size fits all” methodology, he compared service response, network performance, CPU, and internal I/O for each cloud provider.

Here’s a quick summary of the results; you can download the full study — complete with detailed conclusions, test methodology, and even agent code –for free from Webmetrics.

No study like this has been done before – and I hope that many people will take the code, improve it, and run tests of their own.  Oh, and it’s all free.  Here’s the original post on the topic.

Enjoy :)

I’m in a few cities in the US presenting an overview of Complete Web Monitoring for my old alma mater, Coradiant and their partner, Dynatrace. It’s a 45-minute recap of the concept of holistic monitoring, looking at the four kinds of website and linking together disparate sets of data with a particular focus on performance monitoring. Slides here:

If you’re in one of the cities, sign up and stop by. Next stop: Interop, then Web2Expo San Francisco for the Communilytics workshop!

Disclaimer: I’m one of the nominees for the WAA board of directors. Shouldn’t really matter, but consider yourself warned.

This year, 21 nominees are running as nominees for five board of director positions – 1 in Europe and 4 in North America.

I’m running for three major reasons:

• Social Media Measurement: Let’s make sense out of them.  I co-wrote a book on the subject in 2009 with Alistair Croll. I’ve talked to hundreds (thousands?) of practitioners / enthusiasts about the subject. The WAA needs to pursue it’s standardization, advocacy and education in the field.  Yep. I can help that.
• Lean Startup Analytics: It’s time we give more love to the massive startup ecosystem. Startups have completely different needs than companies in the enterprise space. By appealing to them, we can grow our membership base considerably and help educate those that will turn into the mammoth companies of tomorrow.  Think of it as “educating our young”.
• Non-Profit Analytics: Much of the efforts put into strengthening the startup base can equally help non-profits.  By passing along educational material targeted to non-profits, we can empowering them with stronger abilities to  understand where their efforts are successful, and where they’re not.

During the day time, I, eat, poop and sleep data convergence.  We need to be more inclusive of non-traditional streams of measurement. This will ultimately help us grow our member base, and make the organization more relevant in today’s “big data” environment.  This is the central thesis in Complete Web Monitoring.

There are 12 people on the board of directors and the WAA has over 1500 members (updated number from 5000, thanks Eric Peterson).  We have the resources, collective intelligence and drive to make strides in each of these areas.  We simply need a bit of support, encouragement, and direction to make these a reality.  Aaaaand, that’s why I came to the party :).

But frankly, as far as the vote is concerned, I’m confused. It isn’t how I imagined things being. I never imagined that there would be as many existing board members running as there are positions.  The people I’m “running against” aren’t people I want to run against.  I’m supposed to be running against Jim Sterne, the founder of the WAA?

A big part of me wants to give Alex, Dennis, Jim, June and Vicky a second BoD term. Jim puts it so eloquently:

“Aside from being a major cheerleader for the cause, I bring the organizational memory which will help the next Board and our new Executive Director understand why certain decisions were made in the past.”

Another part of me wants to encourage voters to pass the torch on to new candidates armed with fresh ideas and new perspectives.

• John Lovett, analyst extraordinaire, brings an understanding and tenure in the web analytics industry that few nominees can rival.  I encourage the WAA to take the opportunity to work with him closely over the next two years.
• Brendan Hart brings executive metrics knowledge from National Geographic that can greatly benefit our organization from a practitioner point-of-view.
• Eric Feinberg never ceases to amaze me with his candor, knowledge and ability in the field.  He is a great person to talk and work with and would be a big asset to the WAA.
• Steve Jackson, who wrote the book Cult of Analytics, and pushes hard for awareness of the WAA outside of North America.
• And . . . well . . . I’m running and all. :)

I’m not sure which way to go. No matter what, I feel like the Web Analytics Association will gain some great directors, and lose some great candidates.

I’ll probably end up voting for a mix of organizational memory and fresh meat. Either way, the decision will be very hard.

For all of you planning to vote (and even those that aren’t!), I’d love hear your thoughts – either here or on your own blog (if you blog about it, let me know – I’ll add all WAA related posts to the end of this one). By encouraging discussion, I hope that it will help us all come to a conclusion that’s best for the association.

See you on the ballot.

PS: You can see a nominees in action on Twitter by following the Twitter list or finding their individual accounts here.

Here’s a list of blog posts related to the Web Analytics Association Board of Directors vote:

• Vote in the Web Analytics Association Election! by Larry Freed on FREEDyourMind
• WAA Elections: Accountability, Inclusion, and Value by Eric T Peterson on Web Analytics Demystified
• Vote Lovett for the WAA Board of Directors by John Lovett on Web Analytics Demystified
• Big Election Day in Web Analytics Citizenship by Jacques Warren on WAO Marketing

If you haven’t heard yet, the Web Analytics Association is an organization that’s promoting web analytics.  It does so by passing on information to the public, by providing access to best practices, and ultimately connecting individuals, vendors, practitioners and consultants in the field.  This year, the WAA received nominations for their board of directors.  It is accepting 5 positions – 4 based in North America and 1 based in Europe to compliment their existing board.

I’ve put together a Twitter list of all nominees running for the 5 Web Analytics Association board of director positions.  To follow the nominees, simply click on this link, and hit the “Follow this list” button.  Even better, simply add this list if you use Seesmic Desktop, TweetDeck, CoTweet, HootSuite or any other Twitter application.

If you’re interested in following the nominees individually, I’ve included their Twitter accounts below.  Enjoy!

There’s a known weakness in browsers which we wrote about in the book. Every time we talked with someone about it, they’d ask us why we didn’t start a company that took advantage of the loophole, and the answer was, well, it’s creepy. The loophole basically lets you see where else your visitors have been on the Internet. Well, it’s now out in the open, in two forms: Beencounter, and Haveyourfriendsbeenthere.

To be perfectly clear, the site won’t show you everything your visitors surf–just whether or not they’ve been to a set of sites you define. Here’s how it works:

1. You decide what sites you’d like to find out about
2. You embed these sites in a hidden portion of the page
3. When a visitor loads the page, the sites that are visited are marked a:visited in the page’s CSS
4. The Javascript in the page can then grab this property of each link and send it back to you

Knowing where a visitor has been can be used for all kinds of things. For one thing, using just a few sites you can guess the visitor’s gender with a good degree of confidence–resulting in more targeted advertising. This isn’t a new idea (it’s been discussed in terms of browser history before). You might also offer a discount to visitors who’ve already checked out your competition.

Haveyourfriendsbeenthere takes advantage of the obfuscation from a short URL to hide what it is, meaning many people will click on it inadvertently. There’s no easy way to fix this without breaking a lot of the history functions that we use when browsing (one user on Reddit pointed out that this flaw has been around since 2002 and there are sites that show your surfing history already).

We figured it was worth talking about it more openly since these two services are likely to make it a pretty mainstream practice, particularly among sites that benefit from demographic targeting.

BTW, clearing your browser history or surfing in anonymous mode will hide your behavior from such tools.

Beth Kanter‘s 53rd birthday is today.  Of the many reasons why we’re big fans of Beth, she’s pioneering the concepts of web-based analytics for the non-profit / charity sector.  If you haven’t seen her blog before and you want to deep dive in her thoughts on metrics, start here.

To celebrate her 53rd birthday, she’s using social media to incite change in the world by sending 53 Cambodian children to school.  Here’s the full description of her birthday wish.  Have a few bucks laying around?  Help her out here.  It’ll help kids go to school in Cambodia!  How cool is that :).

It gets better.  She’ll write about the lessons she learned during the campaign (just like we did for the Beers for Canada), and share insights on the metrics and measurement tactics she used to determine what worked and what didn’t.

As always, Beth continues to be a rock solid voice in the world of non-profits & web.  She’s on our A-list.

Happy birthday Beth!

Recently, I was in Israel for a cloud computing conference and some meetings with local VCs. The folks at Gemini, a VC firm, organized an evening with their portfolio CEOs to discuss lean analytics for startups. I concluded the presentation with a list of metrics that a web-based startup should track. I guess they were the right questions; at the end of the evening, Guy Horowitz, my host for the event, said,

“I feel bad for the CEOs of my portfolio companies that aren’t here. Their next board meeting will be miserable.”

Not measuring the right things can be fatal. And VCs are in the business of separating the soon-to-be-dead from the fledgling successes. There’s nothing quite as good at doing this as the cold, hard light of analytics. So here’s the list, with a slide deck and some examples.

Message reach

For many startups, the web is their main marketing medium. Because everything online can be tracked, it’s the perfect platform for accountability. All too often, however, we look at metrics like traffic to the site and number of followers. These don’t matter. What matters is your ability to get a message out to people effectively. If you have a cheap way of reaching many people, that’s a valuable asset.

• Your viral coefficient is perhaps the best example of this. Simply put, it’s a measure of how a message will propagate. CEOs and VCs need to know, on average, for each message they hear, how many people does someone tell?

• One measure of this on open social networks is amplification. How well are messages being amplified? This varies by social network: On Twitter, it’s the number of Retweets; on Reddit, the number of upvotes; on Facebook, the number of people who become fans.

Infrastructure health

Many companies overlook infrastructure health, despite considerable recent evidence that page load times strongly influence conversion rates. For an investor, unhealthy infrastructure is often a sign that the startup lacks operational discipline and can point to deeper problems — hacks, kludges, and the kind of duct-tape improvisation that’s fine for the prototype but won’t do in production.

• Performability, which is a combination of performance (latency) and availability (uptime) is usually enough here. But what you measure matters a lot. You need to track the health of key steps in the business process, such as the enrollment loop or the invite process. If you’re a Software-as-a-Service company, you need this data to resolve disputes with subscribers and prove you’d met any Service Level Agreements you’ve commited to.

Market sentiment

You want to know what your market thinks of you. There are many ways to grab this information, but they boil down into two major categories.

• Voice of the Customer surveys ask visitors what they think when they’re on your site.  You can get it from companies like iPerceptions, Kampyle and Foresee. This is crucial in the early stages of a startup, when you’re trying to decide what business you’re in; it’s also vital when evaluating new features, and can act as a safety net when something breaks because a frustrated visitor has somewhere to vent.
• Community sentiment is a combination of search engines and natural language processing. Several startups such as Scoutlabs try to understand what’s being said, where, and how people feel about a particular topic. It’s a hard task to do well because computers don’t understand sarcasm, but knowing when people are talking trash about your brand online is essential.

User engagement

How engaged are your visitors? Ask yourself: How many accounts have you signed up for, then abandoned? If you were to search your inbox, you’d probably find dozens of password confirmations you’ve long since forgotten about. If you’re running a startup, then the chances are good that you’ll be forgotten too. This is an incredibly painful thing to look at. But if it doesn’t make you want to kill yourself, it’ll make you stronger. You can try out new strategies to engage your community and motivate visitors to return, and see if they’ve worked.

• The best way to show this is to measure the time since the last visit by individual user, on a histogram. Here’s an example.

We like this approach because you can see, at a glance, whether people are becoming more or less engaged, as well as how big your “dead pool” of visitors is. We usually map the two on separate axes because your “dead pool” is probably very big — bigger than you expect. Sorry about that.

Lean analytics

Too often, the web data shared at board meetings doesn’t match the business plan. In the business plan of record, you have some assumptions — conversion rates, content a visitor will share, friends they’ll invite. You need to identify these goals, then report them over and over again, with trends.

• Core goals. What are the 3 or 4 assumptions on which the business hinges? If you can’t explain these — and if you don’t lie awake thinking about them every night — find another person to lead the company. This should be the most important thing in your business. Always know what core tasks  you’re hoping your visitors will accomplish, and why they correlate to business growth. If you want to change one of them, make sure your investors all agree to the change, because you’re effectively changing the business model in which they invested.
• Extended funnel abandonment. Each of these goals has a conversion funnel of some sort related to it. Often, the start of that funnel is off on a social network, where someone hears about your site or product. During the achievement of a goal, a visitor probably has to do things that aren’t on your site — opening an email to confirm an address, for example, or authorizing the site to link to Facebook. Each of these steps towards the goal must be tracked. Check out the Product Planner website for some great examples of these steps, along with metrics to watch for each one.
• Movement towards or away from business goals. At every board meeting, you should know what changes to the product moved you towards or away from a goal. If your goal is to minimize calls to tech support, then what changes helped with that and what didn’t? If the startup can’t answer this question, it’s because it’s not experimenting to see which things affect business goals, and that’s a very bad thing.
• Changes to the funnel. Seeing the funnel is useful; seeing it compared to the past period is essential. If you have an extended funnel, and you can see what changed (“email confirmation bounce rates climbed” or “fewer people clicked on the bit.ly link in Twitter”) then you know where to focus your efforts in the coming weeks.

Business sustainability

Once, companies had to decide whether or not to shut down because they were hemorrhaging money. Today, many web businesses can “hibernate”, spending only a few hundred dollars a month on cloud-based hosting while they wait and try to grow organically. That means there are three trajectories: growth, failure, or sustainable burn. Some key metrics will tell you where you stand.

• Cost per engaged visitor is simply the operating costs of the site divided by the number of visitors. It’s how much it costs to provide service to someone. This is a great metric because you can track it month over month, and understand the impact of hiring or capital expenditure on the site. It had also better be smaller than the revenues and conversion rates you’re seeing, or you’re doomed.
• Peak to average ratio. All sites experience traffic spikes. How much busier is the site at busy times — weekends, seasonal holidays, news events, and so on? What premium do you pay to be able to handle those peaks? And can you decommission well? Lots of people worry about scaling up; fewer remember that elasticity is the ability to shed infrastructure quickly and keep things efficient.
• Minimum sustainable burn. If you had to hibernate, what would the steady-state cost of keeping the site running be? This includes salaries, hosting costs, essential services, and so on. Think of it as a disaster plan. If you’re not seeing adoption, and you’re running out of runway, this is the most important number in the company.

It was a great discussion with Gemini’s CEOs, and a good opportunity to discuss what things other companies are tracking. The deck from the Gemini event is available on Slideshare and embedded below.

If you’re a VC or a group of entrepreneurs who’d like to know more about lean startup analytics, bug Sean or myself on Twitter. We’d love to chat.

Alex Bowyer (@alexbfree) is a research analyst at Bitcurrent in Montreal, where he blogs about emerging technologies and their social impacts, and co-organizes events such as Bitnorth and Enterprise Cloud Summit. He is passionate about using computers to solve human problems in new ways, and all the things that encompasses – user-centric design, productivity, human-computer interfaces and exploring social trends. Before that he worked at IBM UK, specializing in Voice systems, Java and information management.

In this post, Alex shows us that sometimes, attracting pageviews isn’t rocket science, which you might be forgiven for thinking if you follow this blog regularly:

Unlike my colleagues Alistair and Sean, I’m no analytics expert. But like all bloggers and social media enthusiasts I have an interest in sharing ideas about technology and society, and getting those ideas out to as many  people as possible.

I’ve been exploring ways to get more traffic to my personal blog, and yesterday stumbled upon something quite remarkable. In one day I was able to achieve 2,579 new pageviews, a 1,176% increase in traffic. And all it took was about 30 minutes of effort.

I’ve been using StumbleUpon’s su.pr as my preferred link shortener for some time, because as well as shortening the URL it allows me to contribute those links to my existing StumbleUpon social bookmark feed while also generating some useful stats. I noticed recently that it makes it very easy for you to “thumb up” and review your shortened links, adding them into the StumbleUpon ecosystem:

[For those who are unfamiliar, StumbleUpon is a social bookmarking service that differs from Digg, Reddit or Delicious in that allows you to click a toolbar button and "Stumble" to randomly selected pages, bookmarked by others, from across all the topics you specify an interest in.]

I also noticed that those links I had submitted to StumbleUpon had gained significantly higher amounts of traffic than those that hadn’t. So as an experiment, I went through every single blog post on my blog and “thumbed-up” the site, assigned it to some appropriate tags and stored it in the best fitting topic. (This last step is vital to maximize the chance of interested eyes landing on your page.)

I waited a day, and I think the results speak for themselves:

What was even more satisfying was to see that these were not just new visitors, but they engaged. Most of the new visitors went on to read other content on my blog, as you can see from the reduced bounce rate,

as well as 1,084 new goal conversions (albeit fairly simple goals such as the amount of time spent on the site):

I think that the reason this experiment was so successful is that people clicking that Stumble button are actively seeking “something interesting” – and are very open to new content and ideas, more so than the average web user. This coupled with the fact that StumbleUpon only directs people to pages for topics they have said they are interested in, results in not just more eyeballs, but the right sort of eyeballs.

I would be fascinated to see this approach used on a site with more measurable goals such as product sign-up or purchases*.

* Disclaimer: Stumblers can rate you “thumbs down” as well as up. Your site reputation may be at risk if you don’t keep providing value to those who rely on it.

Google Analytics recently added a new feature, called Alerts. At first glance, it’s an elegant way to show someone when a KPI on their site has changed significantly from what’s expected. It’s baselining, applied to all KPIs — even the ones you’re not looking at.

This is a great idea for folks who forget to check their analytics data, because now they can find out about significant events. It tricks you into being a better analyst. It encourages baselining, segmentation, and thinking about your business. But we think it’s the start of something bigger, once it incorporates the things Google and others know about your online presence.

Details, and some juicy UI mockup speculation, after the jump.

Baselining, even when you didn’t know you should

Beginner web analysts treat analytics as accounting. They use it to report the news, not make the news. It’s only the more advanced analysts that see analytics as a means for optimization, using things like A/B testing to learn whether a change made things better. And to do that, you need a baseline.

The new feature learns what normal is, then shows you deviation. This encourages experimentation: “I tried something new today, and I can see the results.” Google’s already introduced comparative rankings, showing you how you’re doing against others; now, they make it much easier to identify significant changes to your site, even if you don’t know where to look.

Imagine, for example, that you change your website. You don’t see an appreciable shift in traffic volume, so you decide it didn’t have an effect. But hidden in those traffic numbers is the fact that there was an increase in European traffic at the expense of US traffic. The new functionality would show you this, allowing you to tailor content to specific geographies.

Making segmentation easy to try

The new functionality tries to find chunks of traffic that have “broken away from the pack.” It does this for known metrics and segments — such as geographic regions — as follows:

Notice that little “create segment” at the end? It makes it easy to carve out a slice of traffic you should care about, which then means you can start to play and experiment with it. Segmenting traffic is a sign of web analytics maturity, but until recently, it’s been something few people play with. Now, Google Analytics is essentially telling you, “hey, dummy, have a closer look at this.”

You can use custom segments in lots of cool ways–for example, as the analysis above shows, I now know that returning US visitors are more likely to download content from the site, but first-timers aren’t. Once you’ve seen a segment that Google found for you, you’re more likely to create your own because you understand how they work.

Thinking about your business

You can also set up custom alerts within the system to tell you when something’s gone out of kilter. We know lots of companies who use revenue or transactions per second as the first sign that something’s wrong on the website — this is a great top-down approach if you can manage it, because it means everyone in the company is focused on what actually pays the bills.

The new functionality lets you look for specific occurrences even before they happen. Consider @alexbfree‘s recent post on Twitter Retweeting, which got picked up by Dave Winer. You can set up an alert to see if Dave sends you traffic:

Overall, these are excellent enhancements to the product. They’ll improve engagement — because the system will tell you when things are happening, rather than waiting for you to log in. They’ll encourage good behaviors like baselining and segmentation. And they’ll also satisfy the less business-centric, more hobbyist segment that just wants to know when the world is thinking about them.

What I really want: a holistic view

It’ll be more useful (and in keeping with the Complete Web Monitoring philosophy) when it includes other kinds of data:

• A timeline of posts created, based on Feedburner statistics or blog history
• A series of Google Alerts showing when some search criteria on the web is met
• A volume of followers or friends obtained through the APIs of social platforms like Twitter, Facebook, and Flickr
• Performance data from synthetic or real user monitoring
• Voice of the Customer feedback through systems like Kampyle

Here’s an example of what that could be like, for a content creator/blogger.

That’s a pretty intimidating amount of information. Most of it, Google already has; some, we’d get from elsewhere. We borrowed concepts from:

• Bit.ly’s historical views (over a longer time period) with a rollover for individual links on a given day
• Google Labs’ News timeline
• The dashboard of WordPress
• Postrank‘s content scoring system (we spent time with these folks this week)
• Feedburner RSS stats
• Email subscription management stats from a mailing list provider
• Moni.tor.us performance monitoring
• Trendistic‘s timeline graph of Twitter (with a rollover of Outwit.me‘s realtime tag cloud)
• Google Alerts, which come in by mail but could be turned into a timeline with rollovers

Anyone with a bit of time and some spreadsheet know-how can assemble this manually; it could also be done in Greasemonkey with a bit of work, using Google’s new views as the anchor.

Admittedly, this is still “reporting the news” — the real insight comes from observing correlations, such as what kinds of posts increase subscriptions or what news drives follower count. And this is targeted at a specific kind of site (media/community) whereas other businesses more focused on SaaS or e-commerce revenues probably want something that shows productivity or conversion rates.

Unfortunately, there isn’t a lot of money in giving tools to bloggers. We’re a cheap bunch. So while there’s great multivariate testing for online retailers, a content creator has to cobble together many different views and data sources to paint a complete picture.