It’s been almost 3 months since we launched Mobitest, and we’ve learned a lot in that time. The service is getting used more and more, and it’s always nice to see the phones get hyper-active when we get a mention in some presentation 😉
By now the service is more robust, and we’ve officially removed the Beta mark – coinciding with the Blaze product leaving Beta. At the same time, we gathered up some statistics to share from the tests submitted.
Pages Load Slower In “Real Life”
The first thing we noticed was that the average load time was 6 seconds (without video, more about that later). This is almost 3 times longer than the findings we had in our Fortune 1000 benchmarks!
Sifting through the data, the best theories we have are the set of sites and time of testing. Most tests were run during business hours, when the web is much slower, while our benchmark was run at night, to reduce variability. In addition, many of the sites loaded were lower volume, less sophisticated sites, often not using CDNs and not highly optimized.
Video Adds ~30%
Many of the tests were run with Video, to get a visualization of the load time. We reduced the video frequency to 1 Frame-Per-Second (FPS) to reduce its impact on performance, but we could tell it still takes its toll.
Measuring across many tests, we can now say video adds about 30% to the load time. This stat remains consistent across the different devices. While we still recommend measuring without video if you want more accurate numbers, this helps us know how much adjustment is needed.
iPhone The Popular Client, Google The Popular Site
About two thirds (66%) of tests were submitted on iPhone. This could be impacted by the fact iPhone is first on the list, but it holds even now when we have one iPhone entry and two Android entries… Looks like most users care about iPhone timing more.
At the same time, the most measured URL was google.com. 3.5% of all tests we received were for google.com, most directly to http://www.google.com/, and some to other paths. The runner up was cnn.com, accounting for ~1.5% of tests.
More Granular Percentiles
Now that we have more data from a diverse set of sites, we’ve updated our percentiles and divided them into 10% groups. The following table shows how the percentiles will be divided from now on:
| Percentile | Number of milliseconds |
|---|---|
| Faster than 90% of Sites | 1,539 |
| Faster than 80% of Sites | 2,533 |
| Faster than 70% of Sites | 3,400 |
| Faster than 60% of Sites | 4,323 |
| Faster than 50% of Sites | 5,391 |
| Faster than 40% of Sites | 6,525 |
| Faster than 30% of Sites | 8,208 |
| Faster than 20% of Sites | 11,407 |
| Slower than 10% of Sites | 18,662 |
Note that these measurements are still taken over wifi, and represents runs without video. Now that we know the impact video has, we’ll account for it in our percentile calculation.
We’ll share more info over time, and we have some great features and capabilities in the pipe. To stay up to date follow us on twitter – twitter.com/blazeio