Find Out What Sites Paid To Be Whitelisted From AdBlock Plus
AdBlock Plus has a public whitelist, which is considered to be “non-intrusive” or “acceptable” advertising. However, it seems there is also a private list of Websites that pay to be whitelisted.
I cannot verify those posts and I am definitely not an expert in the subject, but there are clues out there to support these claims This post is just my personal findings so you can come to your own conclusions. The posts are a bit old, so it’s possible AdBlock Plus doesn’t even use this method anymore.
How To Check
From what I can tell, sites use a special IP header to bypass the browser plugin. You just need to run a simple Terminal command to see if the site has it:
curl -I ads.comJust replace ads.com with the domain you want to check. Look for the X-Adblock-Key: entry (highlighted below) followed by a long string of characters, which is presumably unique to the domain.
user@computer ~$ curl -I ads.com
HTTP/1.1 200 OK
Date: Fri, 06 Mar 2015 17:10:00 GMT
Server: Apache
Set-Cookie: gvc=922vr1732074008308404; expires=Wed, 04-Mar-2020 17:10:00 GMT; path=/; domain=ads.com; httponly
Cache-Control: no-store, no-cache, must-revalidate, post-check=0, pre-check=0
Pragma: no-cache
X-Adblock-Key: MFwwDQYJKoZIhvcNAQEBBQADSwAwSAJBAKrfIMFkSaoTSqKmC+BrghK0CpDHc0MuVzmMHin8LIORhpXbped+iYhSnZurWnEO0zcKcVIrzp026LVc5pMB9bUCAwEAAQ==_QhadOD7373K/lmQhzy3NfiEz+luzgToqtfD34RHvdJeiZlHeWUdiNJkLTn7q6NpYtFurMWYji7wXCCOpU/fUow==
Vary: Accept-Encoding,User-Agent
Content-Type: text/html; charset=UTF-8How Does The Command Work?
The curl command is used to download content from Websites and using the -I option returns just the header information from the page.
How Many Sites Have An X-Adblock-Key?
I was curious to find out. So using a simple shell script, I ran the command on every entry in the list of over 120,000 ad domains. The results? Out of 124,735 domains, only about 158 had an X-Adblock-Key.
Why such a low ratio? Most of the domains I tried this on either did not accept the curl command or just timed out, so it’s not a complete list. Many of the domains in the list also no longer seem to exist.
I tried enabling AdBlock Plus on a clean browser and navigated to some sites on the list and they do indeed still show ads.
Most of the domains seem to be designed exactly the same, and many said the domain was for sale from afternic.com. But you can see the list below and come to your own conclusions:
(Some) Domains Using X-Adblock-Keys
More Detail
If you feel like falling asleep, you can read about what X-headers are (or why they should be deprecated). As far as I can tell, they are used for passing non-standard information to get information on a variety of things–Adblock whitelist status being one of them. But again, I am no expert; I’m just piecing together evidence I found.
I would surmise that many of the ad-serving domains do not have a public-facing site, so there isn’t a great way to verify this. As an example, there is no Website for t2.hulu.com, which serves up video ads in Hulu. So the domain exists, but it is a kind of service that you never visually see, which makes it hard to determine if it can be blocked or not.
If you can determine what domain and ad is being served from, you could run the curl command and see if it has an X-Adblock-Key. Let me know in the comments what you find.
Why Is The List Of Domains With An X-Adblock-Key So Small?
A few reasons:
- I set a timeout on the command so it wouldn’t perpetually keep trying to download the header and never complete
- Some domains no longer exist
- Some domains just don’t work with curl or are rejected
- ABP probably has a better way to do this now and has probably stopped using this method
How Can You Block These Sites?
You can try to custom filters in AdBlock Plus but I don’t know if they will be honored or not. You could also us uBlock, which runs with fewer resources than ABP. Or you can try something new: the Raspberry Pi-hole Ad Blocker, which could also be modified to run on any UNIX-like system. This script is what does most of the magic (but you will also need DNS and a Webserver setup)