Adblock Plus and (a little) more

Measuring the memory use of an SDK (Jetpack) based add-on · 2012-01-04 11:37 by Wladimir Palant

Add-on SDK 1.2.1 added a nifty feature: starting with Firefox 9 the memory usage of add-ons built with the SDK is visible in about:memory. However, if you actually try to use this feature you get lost in the sheer amount of compartments. Each module gets its own compartment and the SDK has lots and lots of them. This even prompted a user to report “zombie compartments” caused by my only SDK-based add-on so far.

While there is nothing wrong with these compartments, getting the total amount of memory used by the add-on is everything but trivial. If you don’t feel like adding together a few dozen values you can also go to about:config (or any other privileged page), press Shift+F4 to open Scratchpad and paste the following code:

let prefix = "resource://AddonIdHere-";
let mgr = Components.classes["@mozilla.org/memory-reporter-manager;1"]
                    .getService(Components.interfaces.nsIMemoryReporterManager);
let e = mgr.enumerateMultiReporters();
let n = 0;
while (e.hasMoreElements())
{
  let reporter = e.getNext()
                        .QueryInterface(Components.interfaces.nsIMemoryMultiReporter);
  reporter.collectReports(function(process, path, kind, units, amount, description)
  {
    if (path.indexOf(prefix.replace(/\//g, "\\")) >= 0)
      n += amount;
  }, null);
}
alert("Bytes used: " + n);

Change the value for the prefix variable in the first line to the prefix that is common to all compartments (it depends on the add-on ID, e.g. something like resource://jid0-abcdabcdabcd-at-jetpack if Add-on Builder was used) and press Ctrl+R to run the code. The code will add together all matching memory values and show you the result. For my add-on I got 25 MB on Firefox 9 — quite a proud price tag for 40 lines of code. However, on a Firefox 12 nightly the same add-on takes less than 5 MB so maybe things will improve in future (or maybe it simply depends on how the browser is used, e.g. the nightly build had far fewer tabs open). Note: This probably doesn’t count content scripts, I would guess that their memory use is added to the compartment of the page that they are attached to.

How does this fine-grained memory reporting work, could classic add-ons use it as well? The Add-on SDK uses sandboxes for each of its modules, and Gecko creates separate compartments for such sandboxes. To get this compartment reported properly one only needs to specify sandboxName parameter when creating the sandbox.

Now would it make sense to migrate a large add-on like Adblock Plus to a similar module system instead of the regular JavaScript modules? Or maybe even implement similar treatment for JavaScript modules where each module would get its own compartment? I’m no expert on this but it occurred to me that Adblock Plus regularly exchanges objects across module boundaries. And from what I understood, moving an object from one compartment to another is a non-trivial operation. So while having each module in its own compartment would be nice for memory reporting, it would probably also be a significant performance drawback. I hope that somebody who knows this area better will correct me if I’m wrong.

Tags:

Comment [5]

  1. Zlip · 2012-01-04 16:09 · #

    Is this relevant?, it landed in today’s Nightly x86 update.
    https://bugzilla.mozilla.org/show_bug.cgi?id=677294

    Reply from Wladimir Palant:

    Yes, I think that it is relevant. If I understand it correctly, this change allows creating multiple sandboxes in one compartment. That would mean that one can have a single compartment for the entire extension and load all modules into it – quite an improvement. No more countless entries in about:memory and no more overhead from inter-compartment communication. Too bad that we have to wait 4 months for it. And too bad that this is restricted to sandboxes – getting to that sandbox from UI code or a different extension will be non-trivial, nothing like accessing JavaScript modules.

  2. pd · 2012-01-04 17:31 · #

    That code is relatively simple and short and can be run from unprivileged space. Why on earth then hasn’t something like this been already incorporated into about:memory?

    Perhaps Wladamir you could contribute it yourself? Seems like nobody else – despite the 6-month-long MemShrink effort – is thoughtful enough to do so.

  3. Jeff Griffiths · 2012-01-04 17:38 · #

    The Jetpack project has a bug for this, aptly named ‘Too many compartments!’:

    https://bugzilla.mozilla.org/show_bug.cgi?id=672443

    I think we’re making good progress on this underlying aspect of SDK-based Addon memory usage. Another key change recently was the move to a new module loader design in the 1.4 branch ( to be released Jan. 10 ). Wladimir, I’d be curious to see if there is any difference packaging your addon with the current 1.4 codebase instead.

    Reply from Wladimir Palant:

    At the moment I have 0.5 MB instead of 25 MB – quite an improvement. Still have to see how this changes over time however.

    Reply from Wladimir Palant:

    Sorry, my bad – wrong ID inserted. It’s actually 10.5 MB now. Still an improvement but not quite as drastic.

  4. pd · 2012-01-04 17:55 · #

    Could you give some more instructions on how to use this code? Does Firefox have to be from the SDK prompt? Is there a similar web console way to determine whether installed add-ons were written with the SDK? I could go through all my extensions in my profile manually to look for whichever characteristic that all SDK add-ons share, but is there a better way?

    Reply from Wladimir Palant:

    I’ve changed the instructions to use Scratchpad instead of Web Console – for people who use neither the Scratchpad should be simpler. Note that the code requires chrome privileges which is why you have to open Scratchpad on a privileged page (about:config is one).

  5. njn · 2012-01-07 12:13 · #

    Wladimir, do you have a link to this add-on?

Commenting is closed for this article.