Duplicates in filter subsciptions a burden???

Various discussions related to Adblock Plus development
Post Reply
zerozero7
Posts: 1
Joined: Sun May 24, 2015 1:54 am

Duplicates in filter subsciptions a burden???

Post by zerozero7 »

Please consider the following for the Adblock program if the time required for the benefit/greater good justifies it...
From reading the forums BRIEFLY, something like say "||qrst.facebook.com" may exist in multiple subscribed filters.
I --suspect-- duplicates add a processing delay time/memory size burden for Adblock users.
If not already implemented, could some kind of "summary list" be made & acted on by Adblock where the summary list eliminates exact duplicates as it is being created/appended with each individual filter subscription. This list need not be visible to the individual user, but if someone considers it a worthwhile feature, bonus. "Nearly duplicate" or 'functionally duplicate" like abcd.com^ & bcd.com^ might be taken care of by bcd.com^, the added programming burden for this would be --much-- larger than direct duplicates.

I was thinking about doing this manually by copy/paste/rid duplicates... but this is tedious for individual users to implement, particularly on an ongoing basis .

Please note, that I cannot verify duplicates exist in quantities large enough or the software/user burden from multiple duplicates are large enough that the "additions" benefit/greater good justifies the time & effort of the programmers.

Thank you to all who have contributed to the development & support of the wonderful Adblock software plug-in! :D
User avatar
mapx
Posts: 21940
Joined: Thu Jan 06, 2011 2:01 pm

Re: Duplicates in filter subsciptions a burden???

Post by mapx »

see about the redundancy checker utility (at easylist support forum)
http://forums.lanik.us/viewtopic.php?f=23&t=9041
lewisje
Posts: 2743
Joined: Mon Jun 14, 2010 12:07 pm

Re: Duplicates in filter subsciptions a burden???

Post by lewisje »

I know that Block does something like this after concatenating all the subscription filters into memory, but it only weeds out exact duplicates, not certain filters made redundant by other filters.
There's a buzzin' in my brain I really can't explain; I think about it before they make me go to bed.
harsha_mic
Posts: 1
Joined: Mon May 25, 2015 8:53 am

Re: Duplicates in filter subsciptions a burden???

Post by harsha_mic »

i think from v0.8.9, uBlock is able to remove functional duplicates too rather than just exact duplicates. Since, uBlock now uses Atomic filters during compilation of filters and duplicates are removed during the process..
Post Reply