In this blog post, I’d like to dig into the same data, but this time drill down into the specific types of resources on the page – Image, JavaScript, CSS and HTML files. Such a drill down can give us better insight into which optimizations are being applied today, and help guide us regarding where to focus our performance evangelism.

RWD Page Size

The average size of an RWD site ranged from 1,187 KB on the largest screen to 1,093 KB on the smallest one. The next chart shows those page sizes split into the four primary content types mentioned above, plus an “other” category to capture the rest.

As you can see, the overall size doesn’t change much, which was the point I made in the previous post, so I won’t fret over it too much now. However, the chart captures some new interesting tidbits, which I’ll go over in the next few sections.

It’s worth noting that while I’m only digging into 2013 data here, most of the observations seem to hold true for the data I collected in 2012.

Images make up the vast majority of bytes

As expected, and matching the trend we see across all websites, images make up the bulk of the bytes. Images account for about 6x the average weight of JavaScript files, and 9x that of CSS.

Images are also consistently the biggest component on a page, regardless of screen size. This implies optimizations like Responsive Images, which deliver smaller images to smaller screens, are not being applied.

Only images are optimized to screen size

Across resolutions, the only resource type that truly changes in size across resolutions is image. CSS, HTML, JavaScript and the (tiny amount of) HTML all stay the same across resolutions.

I see this as an obvious sign of inefficiency. Consider the complexity of the “mobile” view (on most RWD sites) compared to that of their “desktop” view. In many cases, the number of interactions or widgets on the page is much smaller, and yet the same amount of HTML, JS & CSS is delivered (and processed) to all screen sizes.

Clearly images are not optimized well enough either, as mentioned in the previous section, and given their size images are probably still the first thing to optimize. However, compared to other resource types, images are miles ahead.

RWD sites are “lighter” than regular desktop websites

Even on the largest resolution, the average page weight in this test suite is ~1.2 MB. This is quite small compared to the overall ~1.4MB average page size, and almost on par with the average home page size of the top 100 websites.

Given the performance issues on the smaller screen, I wouldn’t claim RWD developers are more performance aware than average. However, RWD does force you to think about what’s really needed on your page, which can lead to less clutter on the page – and thus fewer bytes.

And of course there is the possibility the sites inventoried by http://mediaqueri.es/ are not representative enough to compare to the bigger list of websites HTTP Archive tracks. Until we have more known RWD sites we don’t have much we can do about this concern, except take some of the comparisons with a grain of salt.

RWD sites use less JS and more CSS than average

Somewhat surprisingly, only ~10.5% of page bytes (on average) came from JS files, compared to ~15% on websites in general. The gap sounds even bigger in absolute numbers – an average RWD site uses ~120KB of JS instead of the overall average of ~220KB. This may again be attributed to the more minimalist design, or perhaps the need to make these sites work across more old mobile browsers – thus relying less on JavaScript.

However, RWD sites compensate for that with CSS weight, which accounted for 7.5% of RWD site page bytes, compared to 2.5% or less on websites overall. Despite being smaller overall, the average RWD site packs 85KB compared to the 36KB an average website uses – roughly 2.5x more. This part is not surprising at all, as much of RWD relies on CSS, and browser compatibility often requires duplicating CSS content, but it seems to be an area to improve in.

Comparing Requests

Since performance is made up of more than just byte download, this second chart outlines the average number of requests per resolution, again split by content type.

This chart mostly reinforces the insights I mentioned above:

• Total number of requests doesn’t change much across resolutions
• Images make up the majority of requests, by a large margin (though not quite as large as the gap in bytes)
• Average total number of requests – roughly 41 – is quite low compared to the overall average of 91.
• Total number of JS requests is almost half the average number

Unlike what we’ve seen with bytes, even the number of CSS requests is lower than the overall average of 5, though higher than the top 100 average of 2.7. In other words, RWD sites seem to have bigger CSS files, not more of them.

Seeing that the average RWD site uses only 41 requests is encouraging, though it could also imply complex websites don’t feel like RWD is a valid option for them. Another explanation could be that current RWD implementers are – on average – better developers & designers. RWD is still a new trend, and talented teams are more likely to be early adopters, making today’s numbers better than what they’ll be in the future.

Image Optimization – Requests vs. Bytes

RWD image optimization focuses on two main areas:

• Not downloading hidden images
• Downloading smaller images on smaller screens

I was curious to see if websites implement one of these more than the other. To check if that’s the case, I plotted the number of image bytes and image requests side by side, across the different window sizes.

You’d notice the two lines are quite cleanly aligned. This indicates (though it’s hardly scientific proof) that both optimization aspects are equally popular – though calling either popular is a stretch, as most websites don’t apply either.

I would suspect most RWD sites either optimize images or not, but if they do – they optimize both aspects. Helping this theory is the fact most responsive images tools, like PictureFill, also implicitly avoid handling hidden images.

Summary

This data gives us some good further insight into the state of RWD websites today. We can now say that:

1. The push for using responsive images is reaping some fruit. There’s still plenty of work to do before we’re in a good spot, but we’re making progress.
2. Pretty much all other aspects of RWD websites, are not optimized to screen size. We should further evangelize serving just the right amount of HTML, JS and CSS to clients, and build supporting tools.
3. It’s possible that RWD naturally helps reduce clutter and thus make web pages leaner. While not a factual statement, the data we currently have points

Remember that these conclusions, and for that fact all of these data, refer to the state of RWD today. It doesn’t mean responsive websites always behave this way, it just indicates what is actually happening in implementations today. I’ll keep tracking them, to help point out how we’re doing, and where we should focus our attention next.

However, a browser is not just a rendering engine, and the iOS browsers still differ quite a bit. For example, Mobile Safari benefits from a JavaScript engine that is 3X faster than UIWebView. Chrome for iOS, in turn, uses a custom network layer designed to be as fast as the one used by Chrome for Android.

Still, because Chrome for iOS and Mobile Safari are both closed applications, we had no way of actually measuring their performance. We were limited to measuring using UIWebView within Mobitest, and had to assume the other browsers are faster. Fortunately, technology is always improving…

I’ve been working with Manish Lachwani, the CTO of a new startup called Appurify. He and his team have created a platform for automated testing and measurement of black-box apps, on real mobile devices. More specifically, the platform enables some probing into the internals of select apps, such as Mobile Safari and Chrome for iOS, and even lets one simulate network speeds and cellular conditions. Using this technology, we can finally compare the actual performance of these three browsers.

TL; DR;

Here’s a quick summary of the bottom line results.

Overall, Mobile Safari showed to be 18% faster than UIWebView on average. This is a respectable difference, but nowhere near the 3x difference in JavaScript benchmark results. The networking improvements included in Chrome for iOS narrowed this gap, making Chrome only 7% slower than Mobile Safari on average. Despite the small gap, Mobile Safari was consistently faster. It beat UIWebView on 88% of websites and Chrome on 72% of websites – though often by a small margin.

Network conditions significantly impacted the results. On a fast 4G connection, Mobile Safari’s lead grew to 33% faster than UIWebView and 18% faster than Chrome. A similar difference would probably be seen on a good WiFi connection. On the other hand, on a 2-bar 3G connection Mobile Safari was only 2% faster than UIWebView, and it was 2% slower than Chrome. On the average network tier (3G, 5 bars), Chrome beat Safari on 58% of websites, by an average of 5%.

A few key takeaways from these results are:

1. There is a real, quantifiable difference between Mobile Safari and UIWebView-based browsers.
2. However, in real world cellular network conditions, the difference between the browsers on iOS is not huge.
3. The networking layer in Chrome makes a real difference, especially in less than ideal network conditions.
4. The small delta implies using UIWebView-based tools, such as Adobe Shadow, Mobitest, and MIHTool, maintains a decent approximation for the performance Mobile Safari would show, especially under average or poor network conditions.

Short history lesson

About 2 years ago, as part of iOS 4.3, Apple introduced some major improvements to its native browser, Mobile Safari. The most well known improvement was Nitro, an improved JavaScript engine that performs Just-In-Time compilation, making JS dramatically faster.

Unfortunately, apps did not benefit from these improvements. Apple did not include Nitro (and other optimizations) in its embedded browser, UIWebView. Apple stated this was due to security concerns, while others voiced conspiracy theories about Apple crippling HTML5 apps.

Whatever the reason, the embedded and native browsers were not equals, and are still not equal today. When Blaze, my previous company, compared browser performance between iOS and Android, Apple even explicitly stated that the embedded browser does not represent the native browser, due to these missing optimizations.

What the Benchmarks said

Based on benchmarks, the impact is dramatic. JavaScript Benchmark such as SunSpider are 3x faster on MobileSafari compared to UIWebView, implying that browsing a page inside your Twitter or Yelp app would be painfully slow.

Real websites, however, are not benchmarks. We already know that hardware improvements impact benchmarks more than page load times, and can expect the same to be true in this case. In addition, cellular performance isn’t all that great, and if the network is the bottleneck, browser differences may have less of an impact.

Luckily, we’re now able to test these expectations using Appurify, and compare actual load times of real web pages on the different browsers. Let’s get to the results.

Comparing Averages

On average, Mobile Safari was just a little bit faster. When averaging all network speeds, it was 18% faster than UIWebView and 7% faster than Chrome for iOS.

When comparing MobileSafari with UIWebView, there was a clear trend – the slower the network, the smaller the gap. Safari’s lead went from 33% to 12% to 2% as network speeds decreased. This makes sense to me – as the network gets slower, the bottleneck becomes mostly moving bits up and down, and optimizations like a faster JavaScript engine don’t help as much.

The comparison with Chrome for iOS was a bit murkier. Mobile Safari was 18% faster on the 4G network, but it was 5% slower under the 3G 5 bars conditions. The most likely explanation is that Chrome’s touted network layer shines under such conditions. When looking at the poor network conditions case, the delta shrunk, but Chrome was still 2% faster than Mobile Safari.

How did Mobile Safari compare to other browsers?

Safari Consistently Faster

Mobile Safari also outdid the other browsers in the vast majority of tested URLs. Compared to UIWebView, mobile safari was faster in roughly 4 out of 5 sites in the top two speeds, and 3 out of five in the poor network test.

On the top network speed, Mobile Safari also beat Chrome in 80% of cases. However, repeating what we’ve seen in the averages, Chrome did better under worse conditions, and was faster on 58% of websites in both the middle and bottom network speeds, even if the win was usually by a very small margin.

On how many websites was Mobile Safari faster?

How often did it really matter?

Realistically, a 1-2% difference matters very little. If Safari was consistently 2% faster than Chrome, few would call it a faster browser. So, I took a look at how often was the gap between the browsers significant, defined as a 10% or bigger load time delta.

Compared to UIWebView, Safari was much faster when the network was good. An impressive 70% of websites loaded significantly faster on Safari, usually by a gap of 20% or more. Safari was often much faster on a good 3G connection too, but on a bad connection the delta was harder to notice.

When comparing to Chrome, the results repeated the previous pattern. On a good connection, Safari was significantly faster on 60% of websites. That stat dropped to 10% on average conditions, and climbed back to 23% on poor network speeds.

On how many websites was Mobile Safari significantly (10%+) faster?

On the flip side, Chrome also beat Safari by a large margin in a decent number of cases – 29% under poor conditions, and 16% under average conditions. Under good conditions, only 10% of websites loaded much faster on Chrome.

UIWebView beat Safari by a significant difference in less than 10% of tested sites, less affected by network speed.

On how many websites was Mobile Safari significantly (10%+) slower?

Conclusions

My first conclusion is that the delta between UIWebView and Safari is smaller than I thought. Especially under less ideal network conditions, which is when users feel the most pain, UIWebView and Mobile Safari don’t differ much.

Being a big advocate for speed, though, my second takeaway is that this difference is still real and interesting. Especially under good conditions, Mobile Safari is truly faster. This means I still wish Apple would put Nitro and the other optimizations into UIWebView, and believe users would appreciate that.

My third takeaway, and the one I was most surprised to see, is how big of an impact the Chrome networking layer made. Chrome for iOS, despite being built around UIWebView, managed to significantly shrink the gap with Safari – and even beat it often enough. While I can’t say Chrome for iOS is faster than Safari, these numbers make it hard to say it’s slower. This is doubly impressive when you consider this test doesn’t capture some of the predictive optimizations Chrome employs when repeatedly used by real users.

My last conclusion is that the difference is small enough to feel comfortable using UIWebView based tooling. The world of Mobile painfully needs more tools for debugging and testing websites, and these tools only have UIWebView available to them.

Given the results of this study, I believe the browser performance in tools like Adobe Shadow, Mobitest, MIHTool and others is a close enough approximation of Mobile Safari to safely use. This is especially true in poor network conditions, and mostly true in average scenarios, which are the cases where performance matters the most.

Before signing off, I’d like to give another shout out to Appurify, for providing the technology needed to run these tests. They look like a very promising startup, and their platform makes previously complex test cases very feasible. I’d strongly encourage anyone dealing with Mobile to give them a try.

Appendix: Detailed Testing Methodology

As always, I try to be completely open about how I ran my tests. There is no such thing as a perfect test, but by sharing the details you can see any possible bias and use your own judgment on whether you should trust the results or not.

Using Appurify, I measured the page load times of the top 500 websites in the US (per Alexa) with each of three main iOS browsers (Mobile Safari, UIWebView and Chrome for iOS) under three different simulated cellular network speeds (3G 2-bars, 3G 5-bars, 4G LTE). All tests were run using an iPhone 5 running iOS 6.1.2, and were run in portrait mode from San Francisco.

The test flow was as follows:

• For each network speed
  • For each URL
    • For each browser
      • Load the browser with an initial test URL (to avoid capturing app load time)
      • Load the URL being tested (start the timer)
      • Poll the browser every ~30ms to see if it completed loading
      • Once the page is loaded, stop the timer and log the time.
      • Navigate to the test URL again, clean all caches, cookies, history, etc
      • Exit the browser

For each browser, each URL was loaded 7 times, and I used the median load time of those for subsequent calculations. Note that the 30ms interval isn’t always perfect, as timers may invoke a few milliseconds later than planned.

Once I had the per browser data, it seemed obvious that sometimes the browsers did not load the same page. The most common reason was that the page was broken on one or both of the non-safari browsers. Page Size was a pretty good indicator of whether the same page was loaded on multiple browsers, so I only used the URLs where the page size was similar (within a 10%) across all browsers.

The odd broken tests and these filters reduced the dataset to 318 URLs. Furthermore, all the averages mentioned above are calculated while ignoring outliers – the top 5% of tests. Without ignoring outliers, a handful of timeouts and extreme cases threw off the numbers. After ignoring the outliers, the median and averages were very close to each other, and could almost be used interchangeably.

Note that when I compared sites across network conditions, I used the average page load time on each browser (across network speeds), and compared them to each other. This means if a site was much faster under a certain network speed, and a bit slower in other two, it may still be marked as faster overall. This approach isn’t more or less accurate than another, but it’s still worth clarifying.

The last detail to share is the exact network speeds used in this study. I throttled network speeds down to maintain a consistent speed for benchmarking, and used the network speeds that Appurify classifies as Verizon’s averages in different conditions (for San Francisco). Here are the technical details about those speeds:

• 3G, 2 Bars: 193/100 Kbps down/up, 331ms latency
• 3G, 5 Bars: 902/712 Kbps down/up, 150ms latency
• 4G, 5 Bars: 6.8/4.9 Mbps down/up, 29ms latency

If you’re interested in the raw data (they’re slightly complex), you can download it here.

On one hand, images account for over 60% of the page weight. This means they play a major role in overall page load time, motivating dev teams to try and make images as small (byte-wise) as possible. On the other hand, new devices boast retina displays and higher resolutions, and designers are eager to leverage these screens and provide beautiful rich graphics. This trend, along with others, led to a 30% growth in the average number of image KB on a page in the last year alone.

This conflict is partly due to what I think of as “Situational Performance”. If you’re on a fiber connection – like most designers – the high quality images won’t slow you down much, and will give you a richer experience. If you’re on a cellular connection, you’ll likely prefer a lower quality image to a painfully slow page.

Fortunately, not all hope is lost.
A few months ago we created a new optimization called Low Quality Image Placeholders, or LQIP (pronounced el-kip) for short. This optimization proved useful in bridging the gap between fast and slow connections, and between designers and IT, so I figured it’s worth sharing.

Core Concept

LQIP’s logic is simple. In a sense, this is like loading progressive JPEGs, except it’s page wide. There are more implementation details below, but it boils down to two main steps:

• Initially load the page with low quality images
• Once the page loaded (e.g. in the onload event), replace them with the full quality images

LQIP gives us the best of both worlds. On a slow connection, the user will load a fully usable page much faster, giving them a significantly better user experience. On a fast connection, the extra download of the low quality images – or the delay of the high quality ones – does not lead to a substantial delay. In fact, even on the fast connection the user will get a usable page faster, and they’ll get the full rich visuals a moment later.

Real World Example

Here’s an example of the Etsy homepage loaded with and without LQIP. Note that Etsy isn’t actually applying this optimization – I made a static copy of their homepage and applied the optimization to it using Akamai FEO.

On a DSL connection speed, LQIP boosted the page visual by about 500ms (~20%), while on FIOS it was only 100ms faster (10%). This acceleration came from the fact the overall page weight dropped from ~480KB to ~400KB thanks to the lower quality images. All in all, not bad numbers for a single optimization – especially on a highly optimized web page like Etsy’s home page.

DSL Connection, Without LQIP (above) vs. With LQIP (below)

FIOS Connection, Without LQIP (above) vs. With LQIP (below)

The faster visual isn’t the whole story, though. The LQIP page you see actually uses lower quality images than the other. While the LQIP page weighs 80KB less before onload, it weighs 40KB more by the time the full quality images were downloaded. However, the page is definitely usable with the low quality images, keeping the user from idly waiting for the bigger download. You can see an example of a regular and low quality image in the table below – I didn’t turn quality down too far.

Image Before LQIP (15.6 KB)	Image After LQIP (5.2 KB)

It’s just intelligent delivery

LQIP also helps on the political front, by bridging the gap between IT/Dev and the designers.

The designers are happy because their full-quality images are showed to the user, unmodified. Sure, the images are a bit delayed, but the end result will usually show up within a few seconds, and their handiwork would remain unharmed.

IT is happy because they deliver a fast page to their users, even on slow connections. The low quality images may just be placeholders, but (assuming quality wasn’t too drastically reduced) the page is fully usable long before the full images arrive.

Implementation Tips

LQIP implementation includes three parts:

1. Prepare the low quality images (server-side)
2. Load the low quality images (client-side)
3. Load the high quality images (client-side)

Step #1 varies greatly by your system. You can create the images in your CMS systems, duplicate them in your build system, or adjust quality in real-time using tools like Akamai Edge Image Manipulation.

Step #2 is simple – just load the images. You can do so with simple img tags, CSS, or using your favorite scripted image loader. If you use small enough images, you can even inline images (matches Ilya’s recommendations). In Akamai, we use LQIP in combination with loading images on-demand, reducing the number of requests as well.

Step #3 is where a new script probably comes in. A simple flow would be:

1. Create a JS function that iterates the IMG tags on the page, and for each:
   1. Determines the full quality image URL (using a naming convention or an extra attribute on the IMG tag)
   2. Modifies the SRC attribute to point to this full URL (will reload the image)
2. Call your JS function in the onload event

If you want to get fancy, you can load the high quality image in a hidden IMG tag, and then swap the low quality image with it at the onload event. This will prevent the low quality image from disappearing before the full quality image is fully downloaded, which can hinder the user experience.

Lastly, if you use CSS to load your images, you can also swap the low quality images for higher quality images by loading/applying a new CSS.

Summary

I’m pretty excited about LQIP.

It helps bridge the gap between two conflicting and growing needs, would work on old and new browsers alike, and is (relatively) easy to implement. It’s a “perceived performance” optimization, which is how we should all be thinking – and I believe it’s an optimization everybody should apply.

Starting February 16^th, we’ve started using a new data stream, based on traffic from most Akamai customers – meaning a much more global distribution, enough to take away the entire bias. The new data set is also bigger, including over a billion requests each day, and uses a newer version of our device characterization engine.

Notable Changes

Looking at the data, you can see the market share has indeed changed.

For instance, the chart below shows IE’s dominance fell from ~50% to ~40%, while Safari’s share jumped to almost 10%. My guess is that this relates to the better sample even within North America, as opposed to the theory that Safari is just more popular outside the US.

Another example is that Android’s market share increased significantly to over 5% of the overall browsing traffic, and Opera’s share more than doubled. Both browsers are known to be more popular outside the US, which is a likely explanation for this change.

Then again, not everything changed. Chrome, Firefox and Mobile Safari maintained similar market shares, and even a global view doesn’t help make Blackberry seem more dominant…

This new global data set really highlights the need for a geographic split – showing the market shares by continent, country and perhaps even city. We’re working on this feature, along with others, and hope to have it available by May.

Some Kinks to Iron Out

Analyzing a lot of data is not as simple as summing up rows and calculating percentages. Many small details can lead to inaccuracies in the presented data, and we need to make sure we address at least the ones that matter.

For example, Akamai delivers a lot of HTTP traffic that isn’t browsing traffic. If a popular piece of windows–only software made many HTTP requests to an Akamaized domain, it may bias us to thinking windows is more popular than it really is.

Another example is wrongly identifying a device that doesn’t have a huge portion of traffic overall, but on a specific slice of the data (e.g. by geo or on cellular traffic) carries a lot of weight.

In the old data stream, we ironed out most of the big issues, but on the new stream some new ones may have surfaced. We constantly look for these changes, and try to tune the system whenever we find a flaw. If you spot something that looks wrong, please send this feedback to io@akamai.com.

Slow but Determined

It’s been almost 9 months since we launched Akamai IO at Velocity US 2012. When we did, we shared our plans to make it bigger and better over time. These ranged from getting a broader data sample, through adding more types of information, to adding APIs to access the data.

I wanted to clarify that all of these plans are still in place – we’re not moving them forward quite as fast as planned. Know that we are still committed to and investing in Akamai IO, even if we’ve been a bit slow.

Summary

The better data source we’re launching today is a good step forward. We hope you’ll find it useful, and that it will help expand our understanding of the web. You can also expect another good revision in May, and more to follow after that.

Lastly, we want your feedback!

We’ve already received a lot of feedback, and have carefully read and responded to each one –  leading to our current priorities. Please keep the feedback coming by emailing io@akamai.com or through the feedback link on the portal, we truly appreciate it.

The test results didn’t bode well. They showed that practically all RWD websites downloaded the full payload of the desktop website regardless of what was displayed. These websites would download hundreds of KB of unnecessary data to a small screen, only to hide or shrink them on the client, resulting in abysmal performance on mobile devices.

Over the past year, RWD performance has been discussed at length, with great posts from Jason Grigsby, Tim Kadlec and many others. New tools such as PictureFill and Adobe Shadow make optimization and testing easier, and some of the conversation even made it into the HTML5 draft.

So, with all this progress, I figured it’s time for a retest – are we faster yet?

Quick Test Details

Similar to last year, I used http://mediaqueri.es/ as a repository of responsive websites. using WebPageTest, I loaded each of the 471 websites currently inventoried (an increase from 347 sites last year) in a Chrome browser, resized to one of 4 different screen resolutions each time. I then collected the results, and looked at the number of bytes needed to download each page in each resolution.

You can see more details on the test methodology in last year’s presentation, and can rerun the test on a given site with the instructions on this slide.

Problem Solved? Not really.

The results showed that, despite the better tools and conversation, most RWD websites still downloaded the full content of the page on every screen resolution. As you can see from the chart below, RWD pages loaded on the smallest screen were only 9% smaller than those on the largest screen (on average), despite huge differences in the amount of content shown and the physical size of images.

Looking across websites, less than 7% of websites were at least 2x smaller when loaded on the smallest screen compared to the biggest screen. Another 22% weighed between 50-90% of the large-screen page size when loaded on the smallest one, and the majority (72%) were roughly the same size on the smallest and biggest screens.

Not good, but better

While these results are definitely not good, they’re still better than what we saw last year. As you can see in the chart below, the percentage of websites that are much smaller or slightly smaller in 2013 was double what it was in 2012. In addition, the average page in 2012 weighed only 6% less on a small screen than on a big screen, compared to 9% today – again a notable improvement.

This means the enhanced evangelism and improved tooling for RWD performance did have an effect, which is a significant achievement. That said, we probably need this growth to be exponential, not linear..

Unfortunately, while the cross-resolution page size stats improved, the overall performance did not. The HTTP Archive clearly shows how pages are constantly getting bigger, and this trend holds true for RWD sites as well.

The chart below (last one, I promise!) shows the average page size by window size. As you can see, the average page size has increased significantly across all the resolutions. Note that the sites included in each year’s averages are not exactly the same, but this trend holds even if we only look at the URLs shared across the two tests.

Summary

The bottom line is – our work is not done. We’re seeing some fruit of our performance preaching labor, but the vast majority of RWD sites still don’t devote enough attention (or time) to performance. As a result, if you happen to browse a responsive website on your smartphone, chances are it’ll be considerably slower than if you were browsing a dedicated mobile site.

We need to keep pushing and promoting RWD performance, keep building tools and making it easier to be fast, and find a way to improve RWD performance faster than the pace we make pages slower in general…

So for all my fellow performance enthusiasts out there, keep going, we’re not there yet!

Side Note: If you’re interested in the raw test data, you can download the detailed excel sheet with test results, pivot tables and links to the individual tests here.

HTTP 2.0 is in its infancy, but much of its charter is to implicitly get rid of various performance problems HTTP/1.1 presents. Techniques like header compression and request multiplexing try to make websites inherently faster, with no extra effort required of the website owners.

As a result, I often hear statements like “HTTP/2.0 would get rid of the need for FEO”, or even “HTTP/2.0 would make CDNs unnecessary”. I strongly disagree with these statements, and figured it’s worth writing a post about why.

True: HTTP/2.0 would make websites implicitly faster

Making websites faster is a part of the goal with HTTP 2. The IETF is intentionally looking for changes that would tackle real world problems, and make websites implicitly faster. If websites are not faster, the IETF would have failed.

Various optimizations have very little downside. Compressing headers, for instance, always reduces the number of bytes sent and received by the client, and is therefore (practically) guaranteed to offer some acceleration. Multiplexing, as another example, would accelerate the communication with every domain that serves multiple resources on the page. This should accelerate at least one domain in practically every web page.

One caveat to this statement is the requirement for TLS. The use of SPDY currently requires TLS (i.e. requires an https site), which comes with some performance implications. For at least some websites, the performance penalty of switching from HTTP to HTTPS outweighs the value if SPDY, making them more secure – but slower. The current HTTP/2.0 plans don’t include mandatory TLS, but if that changes this will be a concern to keep in mind.

False: HTTP/2.0 would require no changes to websites

Smart websites are quick to adapt to the limitations of the browsers. We change the web page’s HTML to trick the browser into working like we want it to, which often makes pages faster. And then we’re stuck with those tricks long after the browsers are fixed…

There are plenty of examples for this, Here’s one of them: When we saw IE 7 downloads scripts sequentially, we started writing scripts using “document.write()” to make the browser download them in parallel. Many websites still do so today, which in turn makes their sites slower on most modern browsers.

Another example is the use of domain sharding – splitting your content across multiple domains, tricking the browser into opening more connections in parallel. Domain sharding was invented to overcome IE 7’s painful limit of 2 connections per host. On modern browsers, which open 6 parallel connections, its usefulness is less clear, and it often ends up slowing down webpages.

Domain sharding and similar techniques cripple the optimizations of HTTP/2.0. Many of the optimizations in SPDY & HTTP/2.0 are domain-specific, and are most efficient if the content is served from a single domain. To take full advantage of HTTP/2.0 websites would need to – at the very least – undo these optimizations. Other optimizations, like server push, would require new techniques and code to make your website as fast as it can be.

True: HTTP/2.0 client-side adoption will take a while

Sadly, while browsers get updated quickly – many of them fade slowly… Backward compatibility is a very real problem in the web today, and will keep plaguing us long after HTTP/2.0 ships. It took us years to get websites to start dropping support for IE 6, and even today few do so.

On one hand, the fast pace of Chrome and Firefox updates promises browser support for HTTP/2.0 as soon as the draft is finalized, and likely even before. On the other, the slow upgrade process for IE and older Firefox versions guarantees we’ll see browsers that don’t support HTTP/2.0 long after 2014.

True: Automated FEO can make HTTP/2.0 work best

The transition period to HTTP/2.0 will take a while. It’ll be years before the vast majority (say, 80%) of our clients will support HTTP/2.0. In that interim period, website owners will be in a tough spot. Each protocol version would require different optimization techniques, requiring websites to choose who do they love more – HTTP/1.1 users or HTTP/2.0 users.

Luckily, Automated FEO can help with this problem. Since every Automated FEO tool out there (AFAIK) optimizes per browser, and since that tool is the one applying the dirty tricks to tune for older browsers, it’s easy to not apply these techniques for new, HTTP/2.0 clients (assuming the server supports 2.0 as well).

Sticking to the domain sharding example, an FEO tool can easily apply sharding only to 1.1 clients, and skip it – or even undo sharding (consolidating resources into one domain) – for clients using 2.0. In the Akamai FEO tool this is already the default for SSL/TLS connections, and only requires a simple configuration tweak.

We already leverage the fact shared web server software (Apache, Nginx, CDNs) can help the adoption of the HTTP/2.0 protocol itself. Having automated FEO tools can push the 2.0 awareness to the application level, and adjust as necessary.

False: HTTP/2.0 would remove the need for a CDN

Frankly, I don’t know what makes people think HTTP/2.0 would remove the need for a CDN, but I’ve heard this statement from quite a few people. The short statement here is – no, HTTP/2.0 would NOT take away the need to have a CDN.

It’s true that request multiplexing would mitigate the cost of having high latency to your server, but a 20ms roundtrip time (RTT) would still make your page faster than a 200ms RTT. In addition, CDNs bring to fore a huge set of other values, like offloading, reliability, security and more, which are hardly affected by HTTP/2.0.

True: HTTP/2.0 opens up new opportunities for FEO & CDNs

Beyond the fact HTTP/2.0 is not harmful to FEO & CDNs, this new protocol actually opens up new opportunities for these products. When HTTP/1.1 was created, CDNs were in their infancy, focusing purely on caching. As websites changed, both CDNs and browsers evolved, but the channel of communication between them hasn’t, leading to many inefficiencies.

HTTP/2.0 is trying to acknowledge and address common problems and opportunities related to CDNs. Techniques such as browser hints and HTTP-based DNS updates will help websites implicitly get more out of their CDN. Other enhancements, such as server push, open up opportunities for new product offerings CDNs can offer their customers.

Summary

CDNs ♥ HTTP/2.0.

I’m sure many CDNs feel that way, and I can personally attest to the excitement within Akamai. The new protocol rev is an opportunity to get rid of old problems and offer new products, and we’re eager to help shape it and start using it ASAP.

Some early indications of this excitement are the drafts we submitted to the IETF and having Mark Nottingham – the chair of the IETF group handling HTTP/2.0 – rejoin Akamai, and I’m sure there’ll be many more to follow.

As for automated FEO, HTTP/2.0 is all goodness. It presents no threat to its existence, boosts its current value proposition, and opens up a new world of opportunities. This is partly due to the fact HTTP/2.0 is mostly about fixing HTTP, while FEO probably aligns more with HTML. Maybe I’ll be singing a different tune when HTML 6 comes out

I like Responsive Design. Heck, I LOVE Responsive Design. I think it’s a brilliant methodology, which address true challenges in a very good way. But no matter how fond you are of RWD, I think you have to face the music – RWD makes it very hard to write a fast website.

I’m not saying you can’t write a high performance responsive website. I’m not saying you shouldn’t use RWD (Responsive Web Design) – I would actually recommend it to most organizations. However, RWD makes pages inherently more complicated, and all in all would make the mobile web slower.

What’s with all the doom and gloom?

What triggered this post was a great recent post by Tim Kadlec, titled “Blame the Implementation, not the Technique”. The post challenges all these absolute, black and white statements, and says good implementation can address most problems. One of his examples is that RWD websites can have high performance, if you just implement them correctly.

I agree with Tim’s general point. I’ve been talking about the common performance problems with RWD for a while now, and I’m certain most responsive websites out there have terrible performance simply due to poor implementation. I even work on a product that does that sort of acceleration automatically. You can probably make most responsive websites load dramatically faster – often 4 or 5 times faster – on a mobile screen.

However, there’s no escaping the fact RWD – by its very design – puts some real roadblocks in the way of a website trying to be fast.

So what’s the problem?

The problem is complexity.

In the world of the mobile web, the primary two options are a responsive site or a dedicated mobile website (a.k.a. mdot). To be clear, for the purposes of this post, a dedicated website site means a website design purely for a single (or small range) of screen sizes and contexts. For instance, an mdot site design for smartphones.

An average mdot site – a dedicated smartphone-oriented website – is a very simple site. For starters, it usually has a small HTML, that can be quickly delivered in a handful of packets.  In addition, it has very few scripts, CSS files and images, and each of those tends to be small. While mdot sites can have horrible performance due to various mistakes (see this deck for examples), they’re just naturally biased in favor of performance (compared to RWD).

Responsive websites, on the other hand, are complex. A responsive page – when loaded on a smartphone – would still require the browser to download and contend with a big HTML file. After the HTML, the site would need to take extra care to avoid running certain scripts, loading certain CSS and download certain images. Avoiding these resources is possible – and desirable – but it’s definitely not easy. In addition, even if perfectly implemented, avoiding those resources requires code and complexity, and has its own performance price.

You can’t escape this fact. A responsive website tuned to perform the best it can would not be as fast as a dedicated mdot site tuned equally well. Or more realistically, an average responsive website would always be slower than an average mdot site.

Update: It’s worth noting that responsive websites can also use a small HTML with scripts that dynamically generate more HTML on bigger screens. This is a good way to avoid DOM bloat, but it only replaces it with new performance problems created by these  blocking scripts, which delay rendering and get in the way of browser optimizations.

I don’t care. I’m going responsive whether you like it or not.

I know you are. And I would encourage you to do so. As much as it hurts me to say it, performance is not the only thing that matters for web pages. It’s also not the first time the web has gone against the performance track.

If you jump back to the early 90s, you’ll find many websites that are purely textual. These websites would perform AMAZINGLY on any browser. They’ll be faster than Google’s home page, and automatically adjust themselves to a screen on any size – as future friendly as it gets (especially when you consider they were created 20 years ago).

And yet, the user experience on those sites is horrible. The only value in making them fast is that users would very quickly realize they should go to another website. Optimizing your user’s experience must include performance, but I’m not naïve enough to think performance is all that matters, nor would I recommend that approach.

Than what should we do?

We should do what we are currently doing – keep pushing RWD, and make sure we push for performance awareness with it. Push developers and designers to consider performance to be a core requirement, and do the best to address it from day one.  Push browser vendors to make it easier to create fast responsive websites, like the effort to standardize responsive images. Push business to set performance goals and see bad performance as a show-stopping quality bug. On the implementation side, we should do what we can to overcome the main hurdles. Some good places to start here, here and here.

And at the same time, we should accept that RWD means a slower website. Just like Retina images give users richer but slower performance, RWD has its costs. We should accept that as a fact, and do what we can to mitigate the performance cost.

Unfortunately, nothing is really as simple as it sounds… While implementing consolidation is fairly easy, applying consolidation creates two new performance problems: Duplicate Downloads and Breaking Progressive Processing. This blog post explains these issues, and describes the techniques we use – and you can use – to address it in Akamai FEO.

A short disclaimer – the solutions are not trivial to implement. That said, our experience is that they are definitely worth the effort. Besides, if you like the ideas but don’t want to implement them yourself, you can always use an automated FEO solution…

Duplicate Downloads

The most well known problem with consolidation is caching. Consider a website with two pages- a home page, and a product page. The home page uses A.JS and B.JS, and the product page uses A.JS and C.JS. A user browsing the two pages on a site, would download each JS file once, requiring three round-trips to fetch them (one for each of A.JS, B.JS and C.JS).

After applying consolidation for each page, the home page uses AB.JS and the product page uses AC.JS. A user that browses both pages will indeed make only two requests for JS files instead of three, but at the price of downloading the content of a.js twice – once with each consolidated “package”. Since a real web page uses many more JS/CSS resources, a real site has many pages, and pages change quite frequently, redundant downloads happen all the time.

The standard technique to mitigate this problem is to only consolidate “common” resources, shared across pages along a common workflow. This is a trade off – consolidate less (i.e. make more requests), in exchange for downloading less duplicate content. However, striking a perfect balance is very hard, and even if achieved, both problems still remain – you still have some duplicate downloads, and still require multiple requests to fetch resources.

Solution: Adaptive Consolidation

The logical solution to this problem is to split a file into fragments – fetch data into the browser as a single file, but store it in the cache as fragments. Each fragment represents one of the original JS or CSS files (before consolidation). In fact, a fragment would ideally represent a unit of functionality, like a JS module or the styles of a page area.

Unfortunately, browser cache doesn’t work that way, and offers no way to cache a part of a file (MHTML was getting close, but didn’t tackle caching) or to have a script write to cache. Until that’s resolved, we use localStorage instead.

At a high level, Adaptive Consolidation works as follows:

-       At the top of an HTML, “register” all the JS/CSS fragments this page requires

-       Following that, a script checks which fragments are already in localStorage

-       Those that are not in localStorage are fetched with one request

-       Each fragment is evaluated (e.g. execute script or apply CSS) in order

-       Any fragment that wasn’t in localStorage before is now stored (can be deferred till post onload)

I discussed Adaptive Consolidation in more detail in my “DIY Scriptable Cache” Webinar, you can find the slides here and the recording here.

This flow is intentionally simplified, and doesn’t fully match our optimization. In addition, there are some complicated parts in the flow, such as managing the stored data as a scriptable cache; evaluating scripts and CSS with JavaScript without breaking functionality; and handling errors when localStorage is full or unavailable.

Despite these caveats, and as mentioned above, this optimization proved to be extremely valuable to us, completely avoiding redundant data while achieving maximum consolidation.

Breaking Progressive Processing

While it’s well known that Caching and Consolidation don’t get along, few know that consolidation also breaks progressive processing. Let’s go back to the Home Page of the imaginary website from above. This page has two JS files – A.js and B.js – showing up in this order in the HTML. In addition, let’s assume A.js is 1 KB in size, and B.js is 100KBs.

In the unoptimized page, the browser will request both files at once. A.js is small, will likely return faster, and will be executed as soon as it arrives. B.js will be slower, and will execute when its download is complete.

If the resources are consolidated, however, neither will get executed until the combined AB.js file is fully downloaded. To be extra clear here – browsers do not process partial JS or CSS files.

Once again, when you consider the large number of JS/CSS files on an average site, this can be a real problem. Beyond the delay in displaying content, scripts and CSS files often lead to more requests (e.g. CSS images, written tags). These requests will start much later, and in turn the user experience will suffer.

Solution: Streaming Consolidation

As it turns out, there IS one resource that browsers process progressively – HTML. HTML content gets processed as soon as it arrives, which is the foundation for the “Flush The Buffer Early” optimization. So in order to fix the problem above, all we need to do is turn our JS/CSS files into HTML!

For instance, “Regular” Adaptive Consolidation can use a script tag like this:

<script src=”consolidated.js”>

And consolidated.js will hold this content:

Notify(“a.js”,”alert(1)”);
Notify(“b.js”,”alert(2)”);

Streaming Consolidation, on the other hand, will use an IFrame like this:

<iframe src=”consolidated.htm”>

And consolidated.htm will hold this content:

<html>
<script>parent.Notify(“a.js”,”alert(1)”)</script>
<script>parent.Notify(“b.js”,”alert(2)”)</script>
</html>

All in all, it’s pretty easy to convert Adaptive Consolidation into Streaming Consolidation. There is only one catch… it requires Async JS Execution.

The call from the IFrame to the parent is asynchronous, and will therefore not delay other scripts, cannot use document.write(), etc. Implementing Async JS is not a simple feat, so while Streaming Consolidation isn’t hard, it comes with significant pre-requisites.

Summary

Nothing is ever as simple as it seems… Last year I wrote about the problems with taking Inlining too far, and Consolidation is not perfect either. This doesn’t mean you shouldn’t consolidate anything, it just means you should proceed with caution.

If you can afford it, implement or buy one of the solutions above. If not, weigh the issues above when deciding what should be merged with what. And most importantly – measure. The fact you followed a best practice doesn’t mean your page is faster until the numbers tell you so.

A lot has changed since then. Google shipped Chrome for Android, which explicitly stated it addressed the small cache size. Android’s native browser has also improved dramatically, as did Apple’s Mobile Safari. In addition, Firefox released an Android browser, and both Google and Yahoo released hybrid browsers for iOS (Chrome for iOS and Yahoo Axis), which mix Apple’s UIWebView with some of their own software.

With all those changes, plus some probing from Steve Souders, I figured it’s about time to rerun my tests.

Summary Table

Starting from the bottom line, here is a summary table of the max cache sizes per browser and OS, including the new and old results.

Note that the tested Android devices and OS versions were: Samsung Galaxy S (2.2), Google Nexus S (2.3), Motorola XOOM (3.0), Samsung Galaxy S 3 (4.0), Google Nexus 7 (4.1), Google Nexus S (4.1).

As you can see, with the exception of Mobile Safari, cache sizes seem to have grown significantly over the last year. Now on to the details…

Persistent vs Memory Cache

In last year’s tests I attempted to measure both persistent cache and memory cache. Persistent cache meant cache that lives through a restart of the browser process or a power cycle of the device. Memory cache, in contrast, only lives while the browser process lives, and often shrinks if the system requires more memory.

Memory cache proved – then and today – to be extremely unpredictable. Repeated tests showed dramatically different results, with one test often implying a cache size 2-3 times bigger than the other’s. Regardless of whether memory cache is indeed this volatile, or the methodology (explained below) doesn’t measure it well, there’s no point posting these numbers.

For those reasons I decided not to post memory cache sizes this time. Suffice to say I’ve seen cases where memory cache held more than 100 MB of data, and cases where it held less than 10, on the same device. Beyond that, this year’s numbers reflect persistent cache sizes only.

Android Results

For Android, the new “magic number” for a max cache size seems to be 85 MB. This number is used by Chrome for Android on tablets and phones, and in Android’s default browser on both Ice Cream Sandwich and Jelly Bean phones.

It’s worth noting that Tony Gentilcore stated Chrome for Android calculates its cache size based on available disk space, and Matt Welsh mentioned in his Velocity 2012 presentation (Slide #47) Chrome for Android has a cache of 250 MB, though he didn’t state if that was memory or persistent cache.

So while my measurements consistently showed 85 MB of cached data for Chrome (on multiple devices), it’s possible it could go higher.

iOS Results

The biggest disappointment was Mobile Safari. Unchanged from last year’s tests, Mobile Safari has no persistent cache whatsoever. Whenever browser is restarted or the device power-cycled, all browser cache is cleared.

In fact, in iOS 5.1 Apple has even made localStorage non-persistent for apps that use UIWebView (an embedded browser), going against W3C’s guidelines.

However, hope is not lost for iPhone users. In my tests, Chrome for iOS maintained a persistent cache of 200 MB (!!!). It’s possible the cache was even larger, I stopped the test at 200 MB. On a different iOS device, I could never get Chrome to cache more than 35 MB, strengthening the theory that it determines its limits based on available disk space.

To confirm this isn’t just a feature of UIWevView, I tested Yahoo’s Axis pseudo-browser and Twitter’s embedded browser. Both had any persistent cache, making me believe Google wrote some custom code to handle caching – not surprising considering they handle the networking in that browser as well.

Clearly caching is feasible on iOS devices, and it’s possible better caching is one of the reasons many users feel Chrome is faster than Mobile Safari. Hopefully Apple learns from this and bumps up its native browser’s cache.

Other Browsers Results

I also tested two additional browsers – Firefox on Android and the browser Blackberry 9900 (OS 7).

Firefox (Beta) for Android held a 75 MB persistent cache. However, various conversations I’ve had and past browser hacks showed Firefox has a more complicated cache than most. For example, some prefetching techniques showed it holds images and scripts in separate caches. If those caches don’t share a quota, it’s possible the max cache size is bigger.

On Blackberry, the 85 MB magic number surfaced again. This is an improvement from the 25 MB cache on OS 6. That said, it’s not necessarily the future, since Blackberry is now focusing on QNX as their main platform, with a completely new browser.

I did try to test the Playbook’s browser, but my tests indicated it has no memory cache at all. This seems extremely unlikely to me, and it’s easier for me to believe my testing techniques just doesn’t work well on that browser.

Methodology

The detailed methodology is the same one I used in my tests last year. In a nutshell, it primes the cache by loading a sequence of pages (each page loads the next one). Each page holds 4 scripts weighing 256 KB each. All the scripts are cacheable, and the HTTP response of the script sets a unique value to the cookie “A”.

Once the cache is primed, the pages load each other in reverse order, and compare the value of “A” at the beginning and end of the page load. If the resources are loaded from cache, nothing changes the cookie value and it remains the same. If at least one of the resources is fetched over the network, the cookie value changes, marking the end of the cache (with a resolution of 1 MB).

Summary

Mobile browsers evolve quickly, and most seem to have heard the complaints from the web performance community. The average  mobile browser cache size has definitely increased, and hopefully will get even bigger with time. The clear winner is Chrome for iOS, which surprisingly demonstrated a bigger cache than Chrome for Android.

Except for Chrome for iOS, browser cache is still small on mobile compared to desktop.  Desktop browsers claim disk cache sizes such as 250 MB for IE 9, 200 MB for Chrome (based on chrome://net-internals#httpCache) and 540 MB for Firefox (based on about:cache). At least Chrome, and perhaps others, calculate max cache size based on available disk space on both Mobile and desktop, which may account for this gap.

There’s definitely reason to be optimistic about mobile browser cache given these improvements, but we can’t let off mobile browser vendors just yet. I’ll be happy when all mobile browsers match Chrome for iOS for cache size, and think we may just get there after all.

If you haven’t done so already, check out last week’s post to learn about inline vs. offline analysis, central vs. local analysis, and optimizing close to the client vs close to the server. In this post, I’ll discuss the remaining 4 items in the table below.

Resource Origin: Locally Served Resources vs. All Resources

In the last post we’ve discussed where and when to do the analysis, and where to modify the HTML pages. The next question is which resources to include in the analysis and optimization. You can split the resources on the page into two categories: Local Resources, which are delivered by the same system performing FEO; and Remote Resources, which are delivered by a different system, such as 3^rd party resources, resources put on a separate CDN, etc.

It’s pretty clear you’ll optimize local resources, as those are readily available. It’s not clear, however, whether you’ll optimize remote resources. In order to optimize remote resources you must be able to “see” them during analysis. This requires actively requesting them, unlike local resources, which can be passively “seen” as they get delivered. Such active requests are hard to do if you’re analyzing locally, and are even harder if you’re doing inline analysis.

In our ongoing mission to achieve a perfect understanding of the page, we chose to optimize remote resources as well. Remote resources are a big and growing part of web pages today, and without including them in your analysis, your understanding of the page and its performance is very limited. Optimizing these resources means we need to be more careful not to pull in personalized or highly dynamic 3^rd party content, but the results are well worth the effort.

Optimized Resources Storage: Central Storage vs. Local Cache

The decision between central & local analysis also impacts where the optimized resources are stored. If the analysis is done locally, resources are most likely stored in the local server cache. If the analysis is done centrally, resources must be stored in a central location as well, such as Amazon S3 or Akamai NetStorage.

We chose to avoid local cache-based storage because of one word – Volatility.

A cache resource, by definition, is not guaranteed to be there when a request arrives. In addition, having the resource stored locally means the page and all its resources must be served by the same server. In a load-balanced environment, such stickiness is harder to achieve and has significant performance implications.

This volatility has a lot of repercussions.

First of all, every request must include all the data necessary to recreate the resource. This includes the full URLs of the resources, all the conclusions the analysis reached, and much more. This means requests can get very big, especially consolidation requests, adding significant weight to the downloaded page and the uploaded bytes.

Second, versioning is no longer guaranteed. Versioning is the notion of adding a signature (or version number) to a resource, and changing this signature each time the file is changed. This guarantees the same file name will always lead to the same content. If you generate resources on-the-fly, this guarantee is broken. This creates a lot of complications around cache instructions, and can easily lead to the wrong version being cached.

Lastly, some resources may be broken, as not everything can be created on-demand. One example is outlining, where a part of the HTML is moved to an external resource. If that resource disappeared, re-generating it requires fetching the original page again, and even then it isn’t guaranteed. The page would be the same. Another example is the case where two resources must be in sync, like a low-res and high-res version of the same image. Each image can be generated on-demand, but they’re no longer guaranteed to be the same. Each such case has a low likelihood of happening, but they’re far from impossible

There are other implications to using local cache, like the fact that delivering a page and resources requires a lot of I/O activity, the fact certain resources may take a long time to return, and the fact the CPU of creating the resources is duplicated. However, the items above are the primary ones.

Security Model/Data Source: Real Traffic vs. Pulled Content

The discussion above assumes the analysis has access to the HTML page and its resources. This is clearly a requirement, but there are in fact different ways to get this data.

One option is to use real traffic. This means looking at the HTML and resources served to the user, and learning from those. This is probably the easiest way to get access to the website content, and is easier to use when performing inline (or at least local) analysis.

At Akamai we took the other option, which is to explicitly pull the content from the site at analysis time. This is the only way to fetch remote resources, but it can also be used to fetch the HTML and its same-domain resources. This approach is a bit more complex, and may require configuration to access pages behind a login page or in the midst of a step-by-step walkthrough.

This seems like a technical question, but it in fact carries a huge security implication. Every FEO tool creates new resources, which are served to all users. These resources hold content from the original resources, and sometimes even the actual page. If you use real user traffic when creating these resources, you may leak private information. This concern is why we didn’t take this path.

Leaking information is not a likely scenario. It’ll require very specific conditions, like an external JS file with the user’s personal information in it. It’s also likely that properly configuring any FEO tool would prevent this problem.

However, if there’s one thing I learned from working on Web Application Security for a decade, is that if a security problem could happen, it most likely will. If the security concern is not enough, note that auto-generating resources from real user traffic could cause compliance problems with practically any regulation, ranging from PCI to HIPAA.

Upgrade Model: All-in-one vs. Analysis-only upgrades

The last item I want to mention is the upgrade model of an FEO solution. The fact we chose the Central Analysis route offered another side benefit – we can upgrade each part of the system independently. We use a backward compatible protocol between our inline agents and our analysis center, and can therefore upgrade each as needed.

From a delivery perspective, this allows us to upgrade the software as rarely as possible. The inline code mostly applies HTML-aware search and replace instructions, which is fairly simple code, and doesn’t change often. This means less risk is introduced to the component in charge of delivering your website.

From an analysis perspective, this allows us to upgrade the software as frequently as possible. The analysis center is not inline, and if it goes offline briefly, customer websites are not affected. This means we can afford to be more aggressive and deploy new innovative software frequently.

Website owners still need to explicitly enable new optimizations, but this agility makes it easier for us to constantly offer and improve new ways to make their website faster. Between the fast pace of changes in the browser and mobile worlds, and the never-ending innovation cycle, this upgrade model has proven to be invaluable.

Note that if you’re using hosted FEO solution, the upgrade model isn’t really your problem, as it’s a part of the service. However, it’s still worth understanding what happens behind the scenes, and knowing what’s easier or harder for your provider to do.

Summary

As is always the case, building an FEO solution involves more than meets the eye. The architecture decisions outlined above mattered a lot to us, and some of them are the result of trial and error – this isn’t the architecture we originally created…

In fact, FEO architecture is so complex, I had to split this post into two parts, to make it a bit more digestible… And these are just the summaries of the core decisions, not touching on specific optimizations or implementation details!

I hope these points will be useful when you consider using an FEO solution, helping you understand a bit more what are the trade-offs and decisions to make.