How to convert adblock subscription to hosts file

Posting here is no longer possible, please use the forum of a filter list project, such as EasyList
Locked
Jaob

How to convert adblock subscription to hosts file

Post by Jaob »

Is it possible to convert any adblock subscription to hosts file without losing main adblock functionality - blocking all ads included in subscription, or not?

I mean where can be such lines in file as:
&ad_height=
&ad_id=
&ad_number=
&ad_type=
&adCode=
&admeld_
&admId=
&adserver=
&adsize=

And many others. it seems the browser that doesn't has adblock installed (for example on ios devices) won't be able to black all ads.

How can I solve this problem? Will converting ads-sites to 127.0.0.1 block ads in appropriate view?

Please don't recommend me installing adblock from cydia, It's another thread.

Thanks.
lewisje
Posts: 2743
Joined: Mon Jun 14, 2010 12:07 pm

Re: How to convert adblock subscription to hosts file

Post by lewisje »

First, for iOS devices, you can't even modify the HOSTS file without jailbreaking it, and at that point you might as well get Cydia; still, I haven't found an "adblock" program that has been updated to work with later versions of iOS, and they're all poorly coded IIRC.


Anyway, most ABP rules cannot be converted to work in a HOSTS file: The HOSTS file only allows you to redirect requests to particular hostnames; you must use a separate entry for each hostname (wildcards are not allowed, so you can't just use one entry to get "a domain name and all subdomains"), and you can't specify anything more specific, like a path component, port, or scheme.
The closest thing to an ABP rule that can be converted to a HOSTS-file entry is something like ||example.com^ which can become 0.0.0.0 example.com although the latter does not block subdomains of example.com like the former; for example, the Malware Domains subscription can be so converted, because all of its rules are of this form.

The HOSTS file was the original means of mapping hostnames (text-labels representing computers accessible over a network) to IP addresses, before the system of updating it by calling a single computer at Stanford Research Institute was found to be unscalable and then the DNS was developed as a result; it still exists as a legacy component of nearly all devices that can access the Internet, and usually if they find a hostname in the HOSTS file, they just trust it and don't bother with DNS.

You're better off getting a separate HOSTS-file subscription from one of several sources, like Peter Lowe's Ad Servers, or like my own occasionally-updated compilation of several HOSTS files, as described here: https://code.google.com/p/jansal/wiki/HostsFile

Something that may help (and that HostsMan will do for you) is to put up to 9 hostnames on each line, like this:

Code: Select all

0.0.0.0 blocked-domain.net evil.com bad.org malware.info exploit.us crackz.cc hax.cz warez.nu pr0n.tk
0.0.0.0 a.com b.net c.org d.info e.biz f.us g.cc h.nu i.tk
Each hostname after the first is called an "alias" and although some Linux systems can handle more, Windows will ignore all hostnames in the HOSTS file beyond the first 9 on each line; my own pre-compiled HOSTS file, drawn from several sources, is optimized in this manner: https://jansal.googlecode.com/svn/trunk/adblock/hosts


Now for your iOS device, one workaround is to buy the AdBlock Web Browser on the iTunes Store, for $0.99; it blocks about 2500 advertising and tracking domains, possibly making use of Peter Lowe's Ad Servers, and it also allows you to block iframes, scripts, stylesheets, and other elements (albeit on an "all-or-nothing" basis).
There's a buzzin' in my brain I really can't explain; I think about it before they make me go to bed.
Jaob

Re: How to convert adblock subscription to hosts file

Post by Jaob »

Thank you.

And what is the difference between 0.0.0.0 and 127.0.0.1 for blocking sites?
I didn't understand how can I block www.example.com and ssl.example.com. Is it possible to do, without enlarging hosts file three times?

As for jailbroken ios device there is a solution called Adblock and Adblocker but they work very bad and slow down browsing in safari and many other apps. That's why I think there may be another method for jailbroken device without installing waste apps..

As for Adblock webbrowser I found it bad. Because there's no opportunity to add custom rules from txt file. I'm looking for Atomic web browser and it seems rather good.
lewisje
Posts: 2743
Joined: Mon Jun 14, 2010 12:07 pm

Re: How to convert adblock subscription to hosts file

Post by lewisje »

Jaob wrote:Thank you.

And what is the difference between 0.0.0.0 and 127.0.0.1 for blocking sites?
I didn't understand how can I block www.example.com and ssl.example.com. Is it possible to do, without enlarging hosts file three times?

As for jailbroken ios device there is a solution called Adblock and Adblocker but they work very bad and slow down browsing in safari and many other apps. That's why I think there may be another method for jailbroken device without installing waste apps..

As for Adblock webbrowser I found it bad. Because there's no opportunity to add custom rules from txt file. I'm looking for Atomic web browser and it seems rather good.
Thx for pointing it out, I personally disliked the Appbird app's lack of tabs, and this one costs just as much as the one I had mentioned: http://itunes.apple.com/us/app/atomic-w ... 29410?mt=8

The "other method" that you were thinking of is using a custom HOSTS file :D
One good idea is to just download the one I linked to earlier and copy it over your device's HOSTS file; you may need to use a filesystem manager app to do this, and a brief Web search pointed to iFile (available on Cydia): http://iphone.heinelt.eu/?Applications:iFile

The difference between 0.0.0.0 and 127.0.0.1 is that the former is an obviously invalid address, so attempts to go there will instantly fail, while 127.0.0.1 is the local device, so if you're running a Web server on that machine (say for Web development), it's not such a good address to use because while you browse, you'll keep hitting your own Web server; however, this is not such a concern on iOS. If you are able to run a variant of pixelserv (a Web server that delivers a transparent 1x1-pixel gif to every request) on your device, a good IP address to use is the one that the instance of pixelserv uses, which may in fact be 127.0.0.1; to change this in my special HOSTS file, open it in a text editor, press Ctrl+H or whatever the Replace command is, and replace "0.0.0.0 " with "127.0.0.1 " or any other IP address...the reason for the space is so you don't change any blocked domain names that contain 0.0.0.0 inside them, and some of them do...I'd recommend "Replace All" functionality, because my HOSTS file is huge, about half a megabyte.

Before you actually switch over the HOSTS files, you should ensure that you transfer over any special lines from the device's original file; this may not be a problem in iOS, and I know it's not a problem in Windows or on the Mac, but I know in Linux there needs to be a line like 127.0.0.1 localhost device_name and (for IPv6 compatibility) another one like ::1 localhost device_name, or else the machine will have trouble connecting to the Internet.

Finally (doing these in reverse order), you do need a separate entry for each subdomain blocked via the HOSTS file; for example, if you wanted to block example.com, ssl.example.com, and www.example.com, you would need a line like this: 0.0.0.0 example.com ssl.example.com www.example.com
The order of the hostnames doesn't matter for our purposes, but for, say, implementing a round-robin (multiple IP addresses that a single hostname could be mapped to) in a more "intended" usage of the HOSTS file for an Intranet, you would need the lines mapping each IP address to that hostname to be right next to each other, and you can't combine multiple IP addresses on a single line in a HOSTS file.
In summary, you wouldn't quite need to triple the size of the file, but you would need to enlarge it; in this specific example, it's just over twice as large as just 0.0.0.0 example.com
There's a buzzin' in my brain I really can't explain; I think about it before they make me go to bed.
Guest

Re: How to convert adblock subscription to hosts file

Post by Guest »

Does your device have the capability to redirect all its traffic through a VPN? If so, then it would be trivial to setup a transparent ad blocking proxy on a cheap VPS. This would handle any page not sent over SSL.
Guest

Re: How to convert adblock subscription to hosts file

Post by Guest »

The "other method" that you were thinking of is using a custom HOSTS file :D
One good idea is to just download the one I linked to earlier and copy it over your device's HOSTS file; you may need to use a filesystem manager app to do this, and a brief Web search pointed to iFile (available on Cydia): http://iphone.heinelt.eu/?Applications:iFile
It seems there in no alternative browser or tweak for unjailbroken ios device excluding apps you recommended to test. I heard that I can use VPN that will block ads-domains. But can I setup my VPN which will block not only apps but prefixes and sufixxes. That's a controversial question. But it would be a universal solution for every OS.
The difference between 0.0.0.0 and 127.0.0.1 is that the former is an obviously invalid address, so attempts to go there will instantly fail, while 127.0.0.1 is the local device, so if you're running a Web server on that machine (say for Web development), it's not such a good address to use because while you browse, you'll keep hitting your own Web server; however, this is not such a concern on iOS. If you are able to run a variant of pixelserv (a Web server that delivers a transparent 1x1-pixel gif to every request) on your device, a good IP address to use is the one that the instance of pixelserv uses, which may in fact be 127.0.0.1; to change this in my special HOSTS file, open it in a text editor, press Ctrl+H or whatever the Replace command is, and replace "0.0.0.0 " with "127.0.0.1 " or any other IP address...the reason for the space is so you don't change any blocked domain names that contain 0.0.0.0 inside them, and some of them do...I'd recommend "Replace All" functionality, because my HOSTS file is huge, about half a megabyte.
As for IP adresses I understood that blocking domain will block all IP's the website hosted in, isn't it? But there're a lot of examples when some popular sites "were able to be shown" even I manually blocked them in hosts. As an example they are: facebook, twitter. I tried to find the answer but some people are considered to think that such sites use proxies (such as fbcdn.com for xample) and many unknown tools. Sounds disastrously :) Are there any solutions when I can parse a website for IP adresses he allocated with?
Before you actually switch over the HOSTS files, you should ensure that you transfer over any special lines from the device's original file; this may not be a problem in iOS, and I know it's not a problem in Windows or on the Mac, but I know in Linux there needs to be a line like 127.0.0.1 localhost device_name and (for IPv6 compatibility) another one like ::1 localhost device_name, or else the machine will have trouble connecting to the Internet.
Are there limits for hosts file for different OS's? I tested 0.8mb-sized hosts file on iphone 2g and amazigned unpleasantly. it's a common knowledge that iphone 2g is "very fast" nowadays but I didn't expect such slow browsing.
Jaob

Re: How to convert adblock subscription to hosts file

Post by Jaob »

B.U.M.P.
josh522
Posts: 4
Joined: Sat Jun 01, 2013 12:35 am
Contact:

Re: How to convert adblock subscription to hosts file

Post by josh522 »

Jaob wrote:Is it possible to convert any adblock subscription to hosts file without losing main adblock functionality - blocking all ads included in subscription, or not?

I mean where can be such lines in file as:
&ad_height=
&ad_id=
&ad_number=
&ad_type=
&adCode=
&admeld_
&admId=
&adserver=
&adsize=

And many others. it seems the browser that doesn't has adblock installed (for example on ios devices) won't be able to black all ads.

How can I solve this problem? Will converting ads-sites to 127.0.0.1 block ads in appropriate view?

Please don't recommend me installing adblock from cydia, It's another thread.

Thanks.
Unfortunately you can't add filters like that into a hosts file. However, if you want a good hosts file to use on an iOS device, use the on from http://iosadblock.webs.com, I've been using it and i haven't seen an ad since.
Locked