Some more details on Mozilla's add-on performance measurements · 2011-04-06 14:35 by Wladimir Palant

Mozilla Add-ons recently published an article on their effort to recognize slow add-ons. It comes complete with a list of worst offenders. Some media already picked up on that article and users are listening up. So everything is progressing nicely towards the goal — pressure on developers starts building up, I already had something like 10 people point out to me that Adblock Plus is number 12 on that list.

I can however imagine many developers being not too happy right now, and not only because of more attention to the shortcomings of their add-on. Adblock Plus for example already follows all the Performance Best Practices that apply to it, how comes it is supposedly degrading Firefox startup performance by whooping 21%? What is being measured there, how, under which conditions? How much delay are we talking about, in absolute numbers?

So I got Talos source code from http://hg.mozilla.org/build/talos/ and looked around. I couldn’t find the configuration files used to run add-on tests but I could find the raw test results under http://tinderbox.mozilla.org/showbuilds.cgi?tree=AddonTester (you have to go a few days back, the last round of tests ran on April 2nd). So to sum up the findings:

Here are my own testing results on Windows 7 (rounded, these numbers aren’t very precise). I tested various Adblock Plus setups as well as Firebug for comparison (sorry John Barton).

Configuration Time (ms)
Vanilla Firefox 4.0 800
with Adblock Plus 1.3.6 unpacked 900
with Adblock Plus 1.3.6 packed at level 9 (current download package) 915
with Adblock Plus 1.3.6 packed at level 0 885
with Adblock Plus 1.3.6 packed at level 1 895
with Adblock Plus 1.3.6 and EasyList 1160
with Firebug 1.7.0 unpacked 1350
with Firebug 1.7.0 packed 1310

As the test results for Adblock Plus with EasyList (here tested with all the caching introduced in Adblock Plus 1.3.5) indicate, we have bigger worries than whether Adblock Plus is installed packed or unpacked. Having a filter lists adds 250 ms to the startup time for me. Out of these, 200 ms are spent by Gecko parsing the element hiding stylesheet — not something I can really influence. The only way is having EasyList reduce the number of filters somehow (currently it has 11 thousands).

Tags:

Comment [1]

  1. johnjbarton · 2011-04-07 19:05 · #

    Thanks for adding Firebug to your table Wladimir. It’s great to see some real numbers rather than %.

    I guess that bug 639898 can’t really be involved in the startup testing. That would activate Firebug for all pages on startup, that would be quite a bad startup experience.

    One thing I’m curious about: what does startup time mean? I guess for both Adblock and Firebug the addon has to be looking at the initial Web page loads or the user is not getting what they asked for. If startup time means from the time the user launches Firefox until they are reading a web page, then how can any lazy loading make a difference?

    Reply from Wladimir Palant:

    Yes, I forgot to write about that in my blog post – the startup time is measured as the time until a page loads. That’s a file:/// page that will report the time of the “load” event. Which allows for some optimizing on my side. Not really worth it however – other than Talos nobody uses file:/// as their homepage. Real users will be increasingly switching to about:home which is perfect for ABP’s delayed initialization.

    Reply from Wladimir Palant:

    Btw, the actual measurement results for Firebug from April 2nd are 920, 738, 1141 and 901 milliseconds on Windows 7, Windows XP, OS X and Fedora. The corresponding reference times are 542, 401, 700, 496 milliseconds.

Commenting is closed for this article.