Metrics has to be that one component in Web Page Optimization(WPO) that you just can’t afford to get wrong, yet, most people get it wrong. The problem with getting your metrics wrong is that, you think you have fixed a problem, when you haven’t actually. I’ll give an example in each section below.
Before that, a quick summary on why we need measurements in WPO(experts can skip this para).
Assume you have a page (say A) which takes X seconds to load, you do some optimizations on that page A, at this point the only way to tell if your new page (say A1) is performing better than the older page(A) or No is by looking at the new page load time, say Y. Compare Y with X and see if performance has actually improved or degraded. This is the most basic method of looking at performance data and concluding if your optimizations have worked or no.
Which
pic by calliope
Which pages should you measure?
Most of the time, the page that looks obvious is not the one you should be optimizing. e.g If you were to optimize a shopping site, then the Home page is not the one which should be optimizing first, it should be the product listing page or any page which is viewed most.
The key here is to find out which pages are the most visited pages in your site. I’ll give some examples here:
- News site – Article page should be the most viewed page.
- Celebrity site – Photo slideshow/video page should be the most viewed page.
- Listings site – Search results page or listing page should be the most viewed page.
Don’t take my word on this, look at the data you have and find out for yourself.
I think you get the point. There’s no use in fine tuning that page which is viewed by only 2% of your users, pick the one which actually matters and fix it.
All this sounds like a no-brainier right? But guess what, this is the most common mistake most of us make.
Next time somebody tells you their site is slow, this is where you start.
What
pic by practicalowl
Now that you know which pages you have to fix, let’s look at what we need to measure. This part is quite vast and I’ll try to cover as much as I can.
The ideal scenario here would be to stand with a stopwatch behind a person using your site and see know long it took for him to navigate from page A to page B.
Some of us look at full page load time, some look at the onload event, base page load time, Above-the-fold time(AFT) and so on.
What to measure depends on how your page is structured. What is the key selling point of your site, is it a stock quote on top of the page or is it an image in a slideshow or is it an article, or is it user comments.
The easiest way to look at this is – what is it that’s visible in a browser when I load your site. The content that appears above the fold is important, that’s why it’s there(above the fold – the user doesn’t have to scroll to get to real content).
Majority of the sites should be good with just measuring the onload time of a page, because it doesn’t matter what the browser is loading in the background. There are exceptions to this, and that is why I’m saying majority and not all sites(explained below).
This is a good time to look at your page and say, “hey, am I loading things in my page that users really don’t care about or need right away?” Just load what is needed the most first, add the cosmetic stuff later on. Some examples to make this clear.
- Small image carousel in an article page – users come to your site to read the article, so load that article first and load the JS for your carousel after page load.
- Photo page – most of the time, users are not going to scroll down to read comments on each photo, they might just want to see photos. Load the comments on each photo only when a user scrolls to that section of the page.
I can go on and on with this list, but let me get back to the topic we are discussing here.
Exceptions
There are some pages like, a Maps site or a page that does a lot of ajax calls back and forth. Measuring just the body onload time might not be enough in this case. You will have to measure the time it took for the most important content on the page to load. e.g map tile images for a maps site or email in an email application or user comments, etc.In these cases you’ll have to come up with your own set of rules and decide what you should measure.
There has been a lot of discussion on Above-the-fold time and how to measure it, but personally I feel onload time should be the metric to go by for most of us.
Then we have Perceived Performance, this one’s tricky and probably the only one that comes closest to what a human brain perceives as fast.
How
pic by cifor
You know which pages to measure and what to look for in those pages. Let’s look at how we can measure our pages.
How can be classified into a of categories, like, is it a command line script that will be polling a page to measure page load time or is it a browser emulator that is opening a page or is it a piece of JavaScript in a users browser. There are many ways to measure and each one has it’s own advantages and drawbacks.
The first thing we need to understand here is – The user is not going to open the page the way your tool does or have the same Internet connection as your tool does or follow a browsing pattern that you expect. Users are random, totally random.
That is why one should take lot of samples, this evens out the odd ones.
If you don’t have have the tools that simulate your users browser, Internet connection and other such variables, then the best option is to go for beacon based tracking using JavaScript. Use something like Boomerang. This will iron out most of the issues we discussed in the previous para.
WebPagetest is another useful tool that lets you control a lot of variables like browser version, locations, bandwidth, etc. and gives very detailed information.
There are paid alternatives like Gomez and Keynote who specialize in giving very detailed performance metrics.
Where
This section is not relevant for tests that use JS tracking, but it’s still important to keep the where part in mind when you set-up other tests(like WebPageTest).
If majority of your users come from US or any particular region, then make sure your tests are running from that region. There is no point in measuring a page from India when most of its users are from US.
That pretty much sums up what I had to say. There are more topics in metrics that I’d like to talk about, like average vs 90th percentile and more. I’ll save that for another day.
I thought of writing this post because measurement looks like a very small piece of the whole WPO stack, when in reality, it’s not. Flawed metrics not only means waste of time, in some cases it could also mean money.
Do drop a line and let me know what you think about the whole topic of getting metrics right.
