Google Analytics Site Speed vs. Google Webmaster Tools Site Performance Explained

Google is probably the biggest evangelist of speeding up the web, and spends millions each year on making the web faster. More specifically, Google offers users insight into their page load times, using two separate “Real User Monitoring” views: “Site Performance” in Google Webmaster Tools (WMT), and “Site Speed” in Google Analytics (GA).

While both tools serve a similar purpose, the problem is they hardly ever match. The average load times are often dramatically different. On www.blaze.io, for example, GA reported load times over 50% longer than what WMT displayed.

In this post I’ll share my understanding of these tools. Note that WMT is not an open-source tool, and so our knowledge of how it works is based on some experimenting, limited information pieces from Google (plus some from Google employees), and some speculation. I try to point out what is fact and what is theory in the sections below.

Short and Sweet

If you don’t have time to read through, here’s a quick reference table explaining the differences between the two tools:

Webmaster Tools Google Analytics
What is measured Time from Initial Page Request to “Document Complete”
How it’s measured Google Toolbar Navigation Timing + Google Toolbar on IE
Browsers measured Firefox 2-4
IE 6-9
with Google Toolbar & PageRank Enabled
IE 9+
Chrome 6+
Firefox 7+
Android 4+
IE 6-8 with Google Toolbar
Sampling rate All Clients (assumed) 1% of clients, max 10K/day
Averaging techniques 7-day running weighted average (popular pages matter more) 30-day simple average (average the individual page’s average load times)

 

Now to the details…

What they Measure

Both tools attempt to measure the same thing – the time between making the request (clicking a link, choosing a bookmark, etc) and the browser’s “onload” event, when the browser progress indicators stop. This is the same metric used by webpagetest.org and many others to indicate the page is loaded.

In the past, WMT showed the time it took to Googlebot to get the HTML alone, which is how Matt Cutts explains it in this 2009 video. The statements in this video are no longer accurate, Google should probably take it down.

How they Measure

While the two solutions try to measure the same thing, they do it in different ways.

WMT relies solely on the Google Toolbar for measuring page load, getting data from users who also enabled the PageRank feature. There is some speculation that Chrome may have some built in measurements sent to WMT, but I could find no evidence of that.

GA primarily uses the recently standardized Navigation Timing API to collect load times. More specifically, it uses the formula (loadEventStart –  navigationStart). This is a much better understood and more transparent technique. You can see these timings for your page by submitting it on webpagetest.org using Chrome, IE 9 or Firefox as the browser.

GA also uses the Google Toolbar to capture information from IE 6-8, but not from Firefox 2-4.

Which Clients they Measure

One impact of the different measurement techniques is that the two products take measurements from very different clients.

The Google Toolbar is only supported on Firefox 2-4 & IE 6 or newer. It further requires that users enable the PageRank feature, and WMT presumably captures measurements from all users that meet those conditions.

GA’s use of Navigation Timings limits it to newer browsers. This means most Chrome versions, Firefox 7+, IE 9+ and Android 4+ (full list here). Of those, GA samples 1% of users, to a max of 10K users a day (this sampling rate can be slightly increased). Google also gets measurements from IE 6-8 using the Google Toolbar, it’s unclear whether those fall under the same sampling limitations or not.

This difference means the two tools measure very distinct populations. WMT will get measurements mostly from older browsers, and almost entirely from Windows users. GA will sample a much larger set of browsers, though it will still get very little mobile traffic, and not include Safari.

Curiously, this difference should make GA’s numbers smaller (since newer browsers are generally faster), but in reality WMT usually shows a smaller number. Weird.

How they Average

In GA, you have a lot of ways to slide and dice the data you received. The default view shows the last 30 days and includes all pages, showing a simple average of each page’s average load time.

In WMT, the only number shown is the average, and sometimes a few “sample pages”, chosen based on unknown criteria. There is some speculation this is a 7-day moving average, which makes sense given the lack of controls on choosing a date range (they do provide trend over time). As per their docs “The site-level “average” load time is traffic-weighted (i.e., more popular pages have a bigger say)”.

The difference in the time window averaged is only a factor when you’ve made some significant performance changes (like rewriting your code, or implementing Blaze). In that case, you won’t see the impact in WMT for a while (GA lets you choose your time frame). If you haven’t made any major changes, sampling 7 days or 30 days should usually give you similar results.

The weighted vs. simple average, however, is very significant. Most sites have many separate pages, but a small number of those get the bulk of the visits. Those pages often get more attention from development, and are often better optimized and thus faster.

For example, assume your home page loads in 1 second and gets 50% of your traffic, while the other 100 pages all load in about 4 seconds. The weighted load time will be 2.5 seconds, while the simple average will be 3.97.

This difference is the best reason I can find for why WMT often shows smaller numbers than GA.

Which Pages they Measure

GA measures the pages GA was loaded on. This is in your control, and the GA interface lets you easily see and control the pages you’re reviewing.

WMT measures the pages on your site browsed by clients with the Google Toolbar installed and PageRank enabled. It’s hard to say which pages are included in this set, but if your site traffic is big enough, it’s likely to include all pages.

SEO Implications

A key reason these two Google tools have gotten attention is Google’s statements that Site Speed is now a factor in its ranking algorithms. Here’s an excerpt from Matt Cutts’ blog almost 2 years ago:  Google’s webmaster console provides information very close to the information that we’re actually using in our ranking.

While Google never fully exposes such information, this statement implies the number in Webmaster tools is more likely to be the number to care about in SEO. That said, it’s likely that in the real SEO algorithms, page speed is calculated on a page-by-page basis, and not the site average presented by WMT.

Recommendations

Given these differences, GA is clearly superior tool for understanding your page load times. It captures data from more browsers; gives you better tools to drill into specific pages, period in time and other filters; and uses a standardized open technique for measuring.

The main downfall of GA Site Speed is the use of a simple (not weighted) average, so make sure you monitor the performance of your key pages separately.

Keep in mind less than half of the top 10K websites use Google Analytics. Google has to rely on other tools to feed its search algorithms, and WMT seems closer to how those tools may work. Therefore, you should keep an eye on your WMT numbers.

Personally, I find it hard to believe that Google’s search algorithms don’t include information from newer browsers. The WMT numbers are already outdated, and don’t reflect real user experiences, and the gap is only going to grow. I expect more from Google, especially given their dedication to speed.

As mentioned above, we had to deduce some of this in formation instead of getting the straight details from Google. Do you know anything about this topic we didn’t call out? Do you know any of these are mistakes, or missing data? Please share your thoughts and additions in the comments below.

Posted on February 2nd, 2012
  • http://twitter.com/jonathanklein Jonathan Klein

    There’s another reason why the GA numbers are higher than the numbers in WMT – there is a known bug in Firefox’s implementation of the Web Timing spec.  You can read more about it here: 
    http://calendar.perfplanet.com/2011/stop-waisting-your-time-using-the-google-analytics-site-speed-report/

  • http://twitter.com/mdaoudi Mehdi Daoudi

    i am always amazed how everyone loves to speculate how Google does things or think they know how google does it… the truth is that a very few people at google know and they will never tell you. and very few people have access to the data. i worked there and  was in the perf stuff at doubleclick and i begged and begged but no… it’s top secret. 

  • Martin

    Any comment on the relation to the 3rd speed metric from Google?

    https://developers.google.com/pagespeed/

    Pages may perform very well in this metric but may appear slow in GA and WMT..

  • Jagdish

    WMT calls site speed as site performance. It says that ‘xxx site is slower than yy% of sites.
    These estimates are of high accuracy (more than 1000 data points).’

    This data, provides comparative status of site. When it talks about data accuracy it is not clear if site is compared with other similar sites. But it makes sense for Google to compare site speed across some vertical. It would be hard to get this picture using data computed using GA.

    At the end of a day comparative site performance may provide better goals / targets. And may bring useful results. 

  • Leommb

    thanks man for the help was looking for this post  

  • Lemmb

    i have heard that there google has introduced a parameter for recording the site speed which should be placed in google analytics tracking code. this helps give detail report on site speed but i gave not put this parameter in my site (www.collateralmed.com), still GA shows site speed data. Can u please explain how is it fetching the details without adding this parameter?

  • http://www.mobileappsdevelopmentteam.com/ Madt Team

    I am always surprised how people like to take a position how Search engines does factors or think they know how the search engines do it.
    http://www.mobileappsdevelopmentteam.com