Roughly 1 in 8 websites is Responsive

I’ve been on a bit of a quest to find out how many responsive websites are out there.  In a recent post, I describe a decent way to track the adoption rate of the “One Web” concept, but that technique doesn’t work well for tracking responsive design.

Now, I think I found the trick for automatically identifying a responsive site.

The trick: A Horizontal Scrollbar on a Small Screen

That’s it. A simple test, and in fact – as a designer friend of mine pointed out – a technique used for implementing responsiveness before media queries and the term RWD existed.

My proposition is that if you take a non-responsive website and load it in a small browser window, a horizontal scrollbar will show up. However, if the website is responsive, it will magically adjust to the display size, and a scrollbar will not be needed.

I tested this out on a bunch of sites manually, and my theory seemed to hold true. I opened my Chrome window, resized it to be 320pixels wide, and started loading known responsive websites, such as Boston Globe, Starbucks, and the recently responsive Bloomberg. No scroll bar appeared. I tried out various non-responsive websites (plenty of those around), and a scrollbar was always there.

And so, I went ahead and automated a test.

Testing The Test

I wrote a small piece of JavaScript that compares the window.innerWidth value to the scrollWidth of the top elements on the page. If scrollWidth was bigger– the site was marked as not responsive.

I then used WebPageTest’s exec script command to load up a bunch of sites and run this test on them. I used the websites tested by the HTTP Archive as my sample set, as it lists websites in order of popularity across the web.

As a test run, I loaded the top 200 websites on a 480px wide Chrome window and manually vetted the results (I also tried 320px, which produced similar results – more on that in a future post). The results weren’t perfect, but were quite good.

Out of 23 identified RWD sites, only myspace.com, which has an unusual horizontal design, was incorrectly marked as responsive (although reddit has a not very usable panel too). Even more impressive was the fact only one of the 176 sites marked as non-responsive was wrong (tumblr was the exception).

You can check out the screenshots yourself (warning: some sites produced very explicit and NSFW shots)– responsive here and non-responsive here.

The Full Test – 10,000 Websites

For the full test, I ran the same setup on the top 10,000 websites, loading them again in a 480px wide Chrome window. As is common, a portion of the tests failed (due to temporary errors or invalid URLs in the list), making the total number less than 10,000, but close enough.

The following table and chart show the number and ratio of responsive websites for the full list, as well as the top 100 and top 1,000 websites.

top-sites-rwd

# RWD Sites, by Tier Responsive Non-Responsive RWD Ratio
Top 100 11 89 11%
Top 1,000 126 874 12.6%
Top 10,000 1,129 8,102 12.2%

As you can see, the results are fairly consistent, ranging from 11% to 12.6% across the different tiers. Almost 8% of sites failed to load, which is not uncommon when using generic lists.

While I could manually verify 200 URLs, doing the same for 10,000 was a bit much… In addition to assuming the results would be as accurate as the first 200, I came up with one external way to vet my results.

Secondary Verification: Mobile Redirects

In the mentioned recent post, I identified all the Mobile HTTP Archive websites that redirected to a mobile site. Since responsive websites usually replace a mobile site, I would expect to see practically no overlap between the list of RWD sites and the list of sites that redirect to mobile.

Comparing the two lists, I indeed only found 25 websites that were marked as both redirecting to a mobile site and responsive. This is quite small – only 2% of all marked responsive sites sites. It’s also substantially smaller than the 20% portion of all websites that redirect to mobile. Therefore, I see this as another affirmation of the test results.

After manually reviewing the 25 sites, I found a few sites that were mistakenly marked, as well as a few sites that indeed were both responsive and redirected iPhones to mobile, like Scribd, or that had an iPhone specific redirect but not a mobile website, like Opera.

Summary

I’m pretty excited about this new find – both the technique and the data.

Regarding the technique, I feel it’ll be a good (even if not perfect) way to track Responsive Design adoption over time. The data I have so far suggest 95% or more of the sites marked as responsive are indeed responsive, making it accurate enough to use. Beyond tracking adoption, now that we’ve found these sites we can analyze them and find recurring patterns, test performance methodologies and much more – I already have quite a few additional tests on the go…

Regarding the data, I’m actually quite impressed. I was well aware RWD is gaining adoption, but frankly did not expect almost every 8th site to be responsive! I am truly happy it’s getting such broad adoption, and a little freaked out at the performance implications of it, since very few responsive websites make the effort to be fast.

I’d love to get feedback on both data & technique, along with ideas for what to look into next with these new data sets. Please share your thoughts in the comments field or ping me on Twitter.

Posted on December 18th, 2013
  • http://rikschennink.nl/ Rik Schennink

    I feel like a smaller footprint on small devices would be a nice addition to the test.

    In my opinion a responsive site is a site that starts small (with a small footprint) and than additional functionality is stacked on top (and loaded conditionally) based on available features and screen real estate.

    • Guypo

      I agree that’s the ideal, but sadly that’s not the reality. My previous tests showed most responsive websites “weigh” (in KB) roughly the same regardless of the screen size, despite dramatically reducing the amount of visible content.

      See here for more details: http://www.guypo.com/uncategorized/real-world-rwd-performance-take-2/

      • http://rikschennink.nl/ Rik Schennink

        Thanks, nice overview, good stuff! Time for responsive to start meaning more than just scaling things visually, cause that’s only the easy part imo.

  • Jessi Bruton

    Guy, how did you determine the “Top 100 / 1,000 / 10,000″ websites? Are these based on daily traffic, or newness, or some other factor?

    • Guypo

      I simply reused the list the HTTP Archive uses, which in turn is based on Alexa’s Top 1 Million sites, and is based (AFAIK) on traffic.

      Here’s the HTTP Archive description: http://httparchive.org/about.php#listofurls

    • hidrees

      My guess is he used Alexa ranks. I could be wrong.

  • http://www.intersect.com.br/ Elton Mesquita

    Pretty good test and article. To me the number of responsive sites is low, but it’s good to see that it is at least about 10%. Do you plan to repeat this test some more times? It will be nice to compare the growth of RWD.

    • Guypo

      Maybe I’m more of a pessimist, but I didn’t even expect it to be at 10% :)

      Yes, I definitely plan on repeating the test over time, and sharing the results.

      • http://www.intersect.com.br/ Elton Mesquita

        Yep, Point of View… But still it is an ok number.

        It’ll be good to see these numbers through time! I bet we’ll see a big leap in 2014.

  • zachleat

    I’m worried that this test represents a misunderstand of what Responsive Web Design is. Lack of a scrollbar is an oversimplification of RWD. Specifically, a scrollbarless site can be RWD’d but it isn’t a “necessary and sufficient” condition.

    A site with a set of fixed breakpoints is not responsive. If I add media query blocks for 320px, 480px, etc etc and the layout/content is not fluid in-between those breakpoints, it’s not RWD. In the biz we lovingly call this a twerky layout.

    This might be overly pedantic, but I feel like it’s an important distinction.

    • Guypo

      I think the definition of RWD is still somewhat different for different people…
      I agree fluid design is core, but I’m not sure there’s a perfect way to test that. Maybe use a weird width? Like 457px wide?

      I did run 320px wide tests too, I’ll share the comparison of it to 480px once I do some vetting of the data.

      • tkadlec

        Yeah, in actuality the number is going to be even lower than this.

        The reality is the test will ever be perfect short of doing manual analysis (which sounds like LOADS of fun)—there are always going to be exceptions. But I do think you’re on the right track and I’m pretty comfortable with this as at least a rough assessment of the current rate of adoption.

        As for adding a weird width to account for Zach’s concern, I’m not sure how much it would help. Say I serve a 320px fixed layout to a 457px device because my next breakpoint hasn’t kicked in—wouldn’t your test still report that as “responsive”?

        • Guypo

          yeah, it would be marked as responsive in my test.

          You also encounter sites with a responsive menu but not responsive content, or a broken responsive experience (e.g. http://grooveshark.com/), framed sites, and other weirdness.

          I think at the end of the day, we just need to live with a certain margin of error here, from my visual inspection the accuracy is pretty high.

          • zachleat

            The point I was trying to make is that even a visual inspection at one breakpoint is not sufficient.

            You’d need to do a few breakpoints with additional delta tests of +/- 10px (or some other small value). Find an image, test its width. Find the main content, test its width in relation to the viewport.

          • Guypo

            You can find an exception for pretty much any test I create, partly because people expect different things. For instance, your proposed test would intentionally exclude sites that use only breakpoints but not fluid design, while some may call such sites Responsive too.

            Right now, I think the primary question is whether we need to finesse the test. For instance, if you glance through the screenshots of the top 200 sites, can you find any notable number that would require your refinement to accurately identify?

          • zachleat

            Well, there is a standard definition for Responsive Web Design. If we are measuring and collecting data on whether web sites are using Responsive Web Design, shouldn’t our test conform to the definition and not a nebulous approximation of what “some might call Responsive?”

            Anyway, it’s not a big deal. :) Just thought I’d help educate a little bit about what RWD actually is and what it isn’t.

            Happy Holidays

  • g105b

    Quick! Better make my website responsive!
    body { overflow-x: hidden; }
    Done.

    • Guypo

      I don’t think “qualifying” in my test is a dev priority for any site :)
      I am even more certain you’d finding having a usable site on a smaller screen a higher priority than tricking my test.

      If these assumptions prove false, we have a MUCH bigger problem than low adoption of RWD…

  • Dmitry Yesin

    Thanks Guy! This kind of automated detection of responsive sites is something I was working on. One question on using WebPageTest scripting to compare window.innerWidth to scrollWidth. How were you able to output the result of this comparison using WebPageTest? I have looked at the documentation and I can see that you can execute JS code but don’t see of a way to capture the output.

    Thanks!

    • Guypo

      There’s no perfect way to do it – I logged messages into the console (using console.log), and then read them out through the XML Result. I believe it only works on Chrome.

      The best place to ask such questions is on the WebPageTest forum pages.

  • Eric

    Listyouridea.com is responsive check it out