Find Out What Sites Paid To Be Whitelisted From AdBlock Plus

AdBlock Plus has a public whitelist, which is considered to be “non-intrusive” or “acceptable” advertising.  However, it seems there is also a private list of Websites that pay to be whitelisted.

I cannot verify those posts and I am definitely not an expert in the subject, but there are clues out there to support these claims  This post is just my personal findings so you can come to your own conclusions.  The posts are a bit old, so it’s possible AdBlock Plus doesn’t even use this method anymore.

How To Check

From what I can tell, sites use a special IP header to bypass the browser plugin.  You just need to run a simple Terminal command to see if the site has it:

curl -I ads.com

Just replace ads.com with the domain you want to check.  Look for the X-Adblock-Key: entry (highlighted below) followed by a long string of characters, which is presumably unique to the domain.

user@computer ~$ curl -I ads.com
HTTP/1.1 200 OK
Date: Fri, 06 Mar 2015 17:10:00 GMT
Server: Apache
Set-Cookie: gvc=922vr1732074008308404; expires=Wed, 04-Mar-2020 17:10:00 GMT; path=/; domain=ads.com; httponly
Cache-Control: no-store, no-cache, must-revalidate, post-check=0, pre-check=0
Pragma: no-cache
X-Adblock-Key: MFwwDQYJKoZIhvcNAQEBBQADSwAwSAJBAKrfIMFkSaoTSqKmC+BrghK0CpDHc0MuVzmMHin8LIORhpXbped+iYhSnZurWnEO0zcKcVIrzp026LVc5pMB9bUCAwEAAQ==_QhadOD7373K/lmQhzy3NfiEz+luzgToqtfD34RHvdJeiZlHeWUdiNJkLTn7q6NpYtFurMWYji7wXCCOpU/fUow==
Vary: Accept-Encoding,User-Agent
Content-Type: text/html; charset=UTF-8

How Does The Command Work?

The curl  command is used to download content from Websites and using the -I  option returns just the header information from the page.

How Many Sites Have An X-Adblock-Key?

I was curious to find out.  So using a simple shell script, I ran the command on every entry in the list of over 120,000 ad domains.  The results?  Out of 124,735 domains, only about 158 had an X-Adblock-Key.

Why such a low ratio?  Most of the domains I tried this on either did not accept the curl  command or just timed out, so it’s not a complete list.  Many of the domains in the list also no longer seem to exist.

I tried enabling AdBlock Plus on a clean browser and navigated to some sites on the list and they do indeed still show ads.

im4ges adblock bypass

ad bureau ablock bypass

clickdough adblock bypass

Most of the domains seem to be designed exactly the same, and many said the domain was for sale from afternic.com.  But you can see the list below and come to your own conclusions:

(Some) Domains Using X-Adblock-Keys

More Detail

If you feel like falling asleep, you can read about what X-headers are (or why they should be deprecated).  As far as I can tell, they are used for passing non-standard information to get information on a variety of things–Adblock whitelist status being one of them.  But again, I am no expert; I’m just piecing together evidence I found.

I would surmise that many of the ad-serving domains do not have a public-facing site, so there isn’t a great way to verify this.  As an example, there is no Website for t2.hulu.com, which serves up video ads in Hulu.  So the domain exists, but it is a kind of service that you never visually see, which makes it hard to determine if it can be blocked or not.

If you can determine what domain and ad is being served from, you could run the curl  command and see if it has an X-Adblock-Key.  Let me know in the comments what you find.

Why Is The List Of Domains With An X-Adblock-Key So Small?

A few reasons:

  • I set a timeout on the command so it wouldn’t perpetually keep trying to download the header and never complete
  • Some domains no longer exist
  • Some domains just don’t work with curl  or are rejected
  • ABP probably has a better way to do this now and has probably stopped using this method

How Can You Block These Sites?

You can try to custom filters in AdBlock Plus but I don’t know if they will be honored or not.  You could also us uBlock, which runs with fewer resources than ABP.  Or  you can try something new: the Raspberry Pi-hole Ad Blocker, which could also be modified to run on any UNIX-like system.  This script is what does most of the magic (but you will also need DNS and a Webserver setup).

2 Replies to “Find Out What Sites Paid To Be Whitelisted From AdBlock Plus”

  1. Hey,
    Thanks for the interesting article. Perhaps I can help out with a few things. First off, 90% of the entities on the whitelist are there for FREE, but all entities have to abide by the same criteria.

    Second, the whitelist is completely transparent … and available in a much more digestible format here: https://adblockplus.org/forum/viewforum.php?f=12

    Third, OF COURSE users can opt out of it if they don’t like it — and it doesn’t take uninstalling it or special technical knowledge to do so. Actually, it just takes about 2 clicks: https://adblockplus.org/en/acceptable-ads#optout

    Hope that clears some things up!

Leave a Reply