Massive memory usage on Firefox/Linux

[drax]

Hello. I built a VM with slackware linux 14.2. It is a 32-bit VM with 4GB ram. I install Firefox 69.0.3 (32-bit). I then install Ad Block Plus v3.7 for FF.
I do not use Firefox much on this VM. Very light usage. But, I found that Firefox will crash in less than a day. Crash. Restart firefox. Crash next day, etc, etc.

I run Firefox in safe mode. No more crashes.
Then, I re-enable ABP. Firefox starts crashing again after 12-24 hours.

The Firefox errors look like this:

out of memory: 0x0000000000001000 bytes requested
ExceptionHandler::WaitForContinueSignal waiting for continue signal...
ExceptionHandler::GenerateDump cloned child 31994
ExceptionHandler::SendContinueSignalToChild sent continue signal to child
Exiting due to channel error.
Exiting due to channel error.
Exiting due to channel error.
Exiting due to channel error.

Firefox runs out of memory?!? I look at "ps" and indeed it is taking all the memory.
I logged Firefox memory usage with "ps" and plotted it with 1-minute intervals.
Here is the graph:


I start in safe mode; very little increase in memory usage.
Then I re-enable ABP at about 1000 minutes and do some browsing. (see graph) Then Firefox+ABP is idle, but eventually consumes all memory and crashes.
ABP causes firefox to use an additional ~200 megabytes of memory every hour.
What is ABP doing? What does it do every 60 minutes?
With this behavior Firefox+ABP is useless if it crashes the browser every 12-24 hours.

[drax]

I thought I would add more to my thread.

I sat down to my Windows 7, 64-bit computer and found that Firefox (v 66.0.5) was using 12 gigabytes of ram. WTF? I start to investigate this. I used Windows Performance Monitor to log Firefox memory usage. It shows the same pattern as in my previous message. Every hour, Firefox uses an additional ~200 megabytes of ram, DESPITE being totally idle.

I disabled ABP and the incremental memory usage stops increasing. I re-enable ABP and the incremental memory usage pattern returns.

So, this occurs on both Linux and Windows Firefox.

Another thing I tried was entering Firefox's "about:config" menu. I found all the items that had a time value of "3600" or "3600000" and changed them to a different value. This did not change the timing of memory increase. It continued to increase at 60 minute intervals.

Someone must know what this problem is. Or at least how to debug it.

Attached image shows Firefox memory usage on Windows.

[greiner]

Thanks for reporting this. I'm not aware of any other users reporting a similar issue but could you try removing the "ABP filters" filter list from the Advanced tab in the settings page to see whether it solves your issue? Because that particular filter list is updated once per hour in order to respond quickly to any websites that try to circumvent Adblock Plus.

It's worth noting that disabling the filter list is not sufficient since Adblock Plus still checks for updates even when the filter list is disabled. Therefore make sure that you remove it in order to stop it from updating.

[greiner]

There's now an issue for it on GitLab (see core#105) where we track its progress.

[drax]

Thanks for reporting this. I'm not aware of any other users reporting a similar issue but could you try removing the "ABP filters" filter list from the Advanced tab in the settings page to see whether it solves your issue? Because that particular filter list is updated once per hour in order to respond quickly to any websites that try to circumvent Adblock Plus.

It's worth noting that disabling the filter list is not sufficient since Adblock Plus still checks for updates even when the filter list is disabled. Therefore make sure that you remove it in order to stop it from updating.

You have identified the cause of the memory increases. It is caused by the filters updating. But, it does not seem to be just the "ABP filters." I removed "ABP filters" from the list and Firefox still used increasing amounts of memory at 60 minute intervals. I then removed all filters from the list and FF memory usage stopped increasing. I'd think ABP is kinda useless without filters. I reinstalled the filters and Firefox went back to increasing memory usage.

My ABP only had the default set of filters installed. I did not install any extras. Maybe if I play with it more, I could find if there is one filter or combination of filters that causes the problem.

I also tried clicking the button "Update All Filter Lists" and this reproduced the problem. I also tried updating each individual filter by clicking it's gear icon. This too reproduced the problem for each filter. Each update causes an additional ~200 megabytes of memory usage.

I think a work-around for this problem is just to change the update interval. I'd think 1 update per week would be sufficient. No? How often to web sites change their advertising system?

Something else I found is in the Firefox "about:memory" page. If I click "free memory -> GC" button, Firefox will reduce the excess memory usage caused by ABP updates. But, if I click the "free memory -> CC" button, Firefox will not change it's memory usage. I don't know if that tells you anything.

[greiner]

That's good to know. I've mentioned those additional details in the issue report to help us find what's causing the problem.

I think a work-around for this problem is just to change the update interval. I'd think 1 update per week would be sufficient. No? How often to web sites change their advertising system?

Websites and ad scripts tend to change very frequently and we can see the importance of a short update interval by the number of issue reports we receive. Therefore filter lists are updated once every few days.

On top of that, there are multiple websites out there (incl. larger ones such as Facebook) that are actively trying to circumvent Adblock Plus and force ads onto their users. Those sites tend to change much more frequently and therefore require a much shorter filter list update interval.

Therefore we need to get to the root of the problem and fix it rather than increase the update interval, considering that the vast majority of users don't appear to be experiencing this memory issue.

[drax]

That's good to know. I've mentioned those additional details in the issue report to help us find what's causing the problem.

I read the thread on GitLab for the issue report. One member suggested making a before and after Firefox memory report and differencing them. I did that. I could post the memory reports somewhere if you would like to see them. I don't know where to upload them.

It seems most of the memory is allocated in the Main Process's explicit/dom/structured-clone-holder and the WebExtension's explicit/dom/structured-clone-holder.

[greiner]

I read the thread on GitLab for the issue report. One member suggested making a before and after Firefox memory report and differencing them. I did that. I could post the memory reports somewhere if you would like to see them. I don't know where to upload them.

That'd be indeed quite helpful. The simplest approach is probably to respond to the GitLab issue directly and attach the files in the response. However, since that memory dump can include sensitive information, you can run Firefox using a separate browser profile (see Multiple Firefox profiles).

If you prefer not to create a GitLab account, there are various free online storage providers out there such as Google Drive or Dropbox that allow you to upload and share files with others. Those give you a link that you can share either in public here or in private via email to me at thomas [at] adblockplus . org.