Do faster web pages mean better business? Definitely. We’ve seen hard evidence from major web operators like Shopzilla, Google, and Microsoft. But what about other websites? How big an impact does performance optimization have on the business metrics of a typical media or e-commerce site?
Here’s some concrete data on how reducing latency changes the key metrics, such as bounce rate, pages per visit, conversion rate, and shopping cart amount. It’s a pretty detailed discussion, but it if you want to understand the ROI of improving web performance on your site, dig in. If you want to read this more easily, here’s a PDF.
Enough theorizing: Page latency affects your business
Years ago, the need for web performance was anecdotal. Researcher Mihaly Csikszentmihalyi had, years before, shown that human beings are more engaged, and more likely to enter “flow states”, when response to their actions is immediate (see this article for more details on flow and web optimization.) But while it seemed like a good idea to make a page load quickly — and companies like Zona Research made headlines with their 8-second rule — we didn’t have any empirical evidence to that effect.
There’s no longer any debate. There’s reliable, reproducible evidence that web page latency is directly tied to the bottom line. At Velocity, Microsoft, Google and Shopzilla made this abundantly clear in a series of awesome presentations: detailed, controlled testing proves that slower pages hurt the bottom line. In Google’s case, adding delay reduces the average number of searches a visitor does each day even after the delay is removed.
Microsoft, using data from Bing, showed that slow pages affect other KPIs:
In other words, if your website is slow you’ll get:
- Fewer search queries per user
- Less query refinement
- Less revenue per visitor
- Fewer clicks, and lower satisfaction
- A longer time for visitors to click something
- Fewer searches per day
- Lower search engine rankings
Ouch.
There’s a great writeup of all this at Artzstudio, and we covered Marissa Mayer’s talk on Bitcurrent. My only complaint was that the sessions all left me with questions about how this applies to smaller, more targeted sites.
So what about sites we mortals run?
Generally speaking, there are a few KPIs that really matter. These measure attention (how many people find out about you), engagement (how much they interact with your site), and conversion (whether they do what you wanted, and how much you benefit from it.) Here’s a simple view of some of those KPIs. Different KPIs matter as visitors turn into customers, buyers, or enrolled users.
In this diagram, inbound traffic either consists of returning or new visitors. Some of those visitors leave (“bounce”) immediately, while others continue on to other pages, spending time on the site, creating content, and viewing ads. On an e-commerce site, a subset of those visitors convert (by doing something we want them to) and there’s a value to those conversions.
The question, then, is how does a faster page load time affect metrics such as these?
Comparing performance optimization in analytics
At Interop Las Vegas, I had dinner with the guys from Strangeloop Networks (disclaimer: I have friends who work there). They make a web acceleration appliance that speeds up page load times. It occurred to us that they were in a perfect position to tie page performance to analytical results, because their appliance could actually modify pages on their way out, on a visitor-by-visitor basis. So if they only optimized some of the visits, and marked them as such, they could later compare the business performance of optimized and unoptimized segments.
The Strangeloop guys ran sprinted with this idea, testing a couple of different types of site, including media and transactional businesses. Here’s what they found, which they’ve agreed to share with us provided we keep certain details confidential.
Total visits
The system was set up to optimize half of the visits and leave the other half untouched. But when they analyzed the results, that’s not how things looked in Google Analytics, where significantly more optimized sessions were recorded. In all, roughly 14,000 visits were segmented within the analytics package; but the number of optimized sessions that were recorded was significantly higher.
This may be due to problems with the analytics scripts running on slower connections, or it may be a sign of increased page abandonment before analytics has a chance to load. Either way, it’s worthy of more study.
Percent of visits from new visitors
Optimization seems to have an effect on the number of new visitors to the site, too, though it’s not clear why this happens.
We’d expect that since new visitors’ browsers have more to load — there’s nothing in their cache yet, and so page load times are higher — there would be fewer aborted visits for new users, which in turn would increase the relative number of new visitors to the site. But that’s not what happened.
Bounce rate and optimization
The next metric we looked at was bounce rate, that is, how many visitors left quickly from the first page they saw. Faster pages delivered a lower bounce rate than slower ones.
This seems intuitive, since when a page loads quickly you’re less likely to leave out of frustration. That extra percent of visitors who stick around turns into more opportunities to sell something or otherwise engage a visitor.
Engagement: Pages per visit and visit duration
Optimization also affects the number of pages a visitor views. When the site is slow, people read fewer pages.
It’s not clear from the pages-per-visit metric alone whether this is because people spend a finite time on a site, and simply get bored after that time, If that were the case, we’d expect a faster-loading page to result in more pages per visit simply because more pages could be loaded before the visitor got bored. A second KPI, Average time on site, clarifies this. Optimized visitors to the site spent 7 minutes more than unoptimized visitors. So it’s not so much the pages per minute of time on the site, but the actual number of minutes, that increases.
This is great for visitor engagement; if you’re running a media site, it also means a chance to deliver more impressions and make more money per visitor.
Impact of optimization on e-commerce results
But what about e-commerce and retail? Strangeloop instrumented a second site in the same way. The beauty of tagging visits up front is that the actual business outcome of that optimization can finally be quantified. In this case, optimization resulted in a 16.07% increase in conversion rates and a 5.50% increase in average order value.
We can’t share the actual conversion rate and order value amounts, but what these numbers do is allow you to actually quantify the ROI of a performance improvement investment.
Want to learn more?
There’s lots of good data behind these results, which we’ll be looking at in more detail in a Webinar on October 8 at 2PM EDT (you can sign up online at bit.ly/perfwebinar). Strangeloop will also have some data on how much performance improvement visitors experienced by then, and you can ask Hooman Beheshti (their VP of Products) and I questions about the experiment if you want to know more.
Some caveats:
I’m always wary of presenting vendor-specific data, because we try to remain impartial. Strangeloop isn’t paying me to talk about this, and I decided to cover it because it’s useful to the web monitoring community and I asked the question in the first place. I’ve reviewed the information fairly closely and have good reason to trust Strangeloop (for one thing, their VP of products, Hooman Beheshti, is a sometime contributor to Bitcurrent and an expert on web performance who reviewed Complete Web Monitoring and gave us detailed feedback.)
Instrumentation happens as follows:
- Every visitor who requests a page gets a segmentation cookie regardless of who they are. Since Strangeloop’s technology takes advantage of certain features in more modern browsers, not every visitor who is accelerated will benefit from the same performance improvements.
- The numbers reported in the analytics package (Google Analytics) are a result of the segmentation cookies seen, which is tied to how many visitors’ browsers made the analytics request.
- The reporting Javascript within the page sometimes can’t find the cookie that was set (this happens about 5% of the time). This could be a consequence of security restrictions, browser limitations, and so on. However, the “cookie not found” errors occur relatively evenly across optimized and unoptimized visitor, so they don’t distort the numbers.
- Google Analytics’ IP filtering was used to block out internal users, which might distort numbers too.
Related posts:
Great stuff Alistair.
Did you know that if you place Google Analytics on the bottom of the page – the numbers can be drastically different to if they were deployed at the top of the page? TagMan's study on the latency of analytics / tracking tags based on placement showed a 20% difference. The study is here: http://blog.tagman.com/?p=189
I would also expect to see a difference in Browser Type results here – e.g, Internet Explorer's Java engine much slower than Chrome – therefore Ajax pages load faster = less bounce for certain site types.
Thanks for the read!
Chris:
Browser factors shouldn't matter because they'd be spread across both groups evenly (or at least I hope so).
Also I would assume that they used the same GA tag implementation across the site, and therefore it doesn't matter because optimized and unoptimized got the same treatment (unless the Strangeloop Appliance mentioned) is shifting the javascript around on the page (which isn't totally out of the question).
Hmm – seems like my comment above reads more as a critique of the study – rather than suggestions on points to consider looking at ON TOP OF these results, which is what the comment was meant to read like! This is a great piece.
In the Strangeloop experiment how many ms faster was the optimized than the non-optimized version?
Chris/Adam,
Yeah, tagging was controlled in both groups (i.e. while improper tagging of pages is a bad thing, and moving the Javascript around can help load the page so the tags load in the background, it wasn't a reason for the differences in this experiment.)
While moving Javascript to the bottom of the page is often a best practice, for many sites that are dynamic, it's not an option — the basic mechanics of the page require it. Also, browser enhancements in Chrome, Safari, and Firefox's latest Javascript engines have dramatically sped things up.
What does slow things down is the retrieval of external objects, particularly when they come from other sites whose URL must be resolved with a DNS lookup.
My question too. The short answer is, I hope to find out more next week with their VP Products on a webinar (the link is in the body above) but the long answer is it depends. Strangeloop does a bunch of things to speed pages up — many of them similar to the best practices Steve Souders recommends as part of ySlow — automatically. There's also a bunch of secret sauce. But many of these depend on how recent the browser is, since newer browsers have more advanced functionality that can be taken advantage of to squeeze performance from them.
Ideally, we'd know exactly how much conversion improvement we got for different performance improvements, and fit them to a curve. Then calculating actual ROI would be easier.
Superb, hopefully you will post your findings from the VP, here?
I know it might be hard to come up with exact figures, as you point out, e.g. it is browser dependent. But a relative browser dependent number is better than none.
I have a lot of upcoming conversion and usability testing right now, but I have written “curve on relation between load performance and conversion” on my long list of future conversion experiments. Lets see if you beat me to it :)
Hi Alistair,
A really great article, and kudos to the Strangeloop guys for getting their customers to run some tests on their users! I assume the customers were aware of the tests ;-)
This obviously bears out what our customers have been telling us, but it is extremely useful to have this kind of data available to point people to. The methodology is a good use of existing tools, and (as an aside) can be replicated in our product
http://knowledgehub.zeus.com/code/2009/10/01/me...
Nick
Thanks for the kind words, Nick. We're glad you liked the article. We're huge fans of independent benchmarks that show actionable results – everybody wins that way.
Personally, I'm a big fan of Zeus. A number of years ago, while working at Coradiant, I led an Interoperability Lab. I had the task of testing network taps and their behaviors under situations of extreme load. Coradiant owns the findings, but it turned out that Zeus could handle the most amount of load with the least amount of setup.
I've been a fan ever since.
Thanks again for stopping by.
Great experiment- what features were applied to the “accelerated” traffic?
There are a few automated solutions to speed up your website. For PHP it's Web Optimizer – http://www.web-optimizer.us/
Great article. Solid metrics have been hard to find.
Great article. Solid metrics have been hard to find.
well i always thought faster loading sites would help convert better, are there any affordable private servers you suggest?
Very helpful information. I bookmarked this article. Thanks for sharing.
Jessica S.
http://nocostwebsite.co.cc/
A slow site can have a big bounce rate. In other words, a visitor, a possible client, that tries to access the site looking to buy a product, might leave the site because it isn't loading fast enough. That's just one example of how a slow site can damage one's business.
Mathew Farney | Trianz
Personally, I'm a big fan of Zeus. A number of years ago, while working at Coradiant, I led an Interoperability Lab. I had the task of testing network taps and their behaviors under situations of extreme load. Coradiant owns the findings, but it turned out that Zeus could handle the most amount of load with the least amount of setup.
I have never used Zues, where to get?
I think life is simple and comfortable, so the decorate at my home is very concise. This is also my character, my life is like this, these simple store are my favorite. Nike air Zoom Lebron soldier IV
The post about leisure is quiet good. Reduce our work pressure, improve our life mood . I like it very much.I will support it often. These days I want to buy something from these websitesXXXXXX,but I don't know how to do.I hope you can help me.Thank you very much! Nike Zoom Blur Low
Nice work.
I like Jimmy Choo this brand，such as Jimmy Choo Sandals are promotion now,up 79% off.Jimmy Choo Shoes have fashion style now.