pihole

Improving The Pi-hole's Performance For Blocking Ads

Jacob Salmela

Jun 6, 2015 • 10 min read

Due to the last update of one of the sources the Pi-hole gets its ad domains from, performance of the Pi-hole took a big hit, particularly on the original Pi B+. If your Pi-hole was affected, just download this list from before the update using the command below, and it should return to normal behavior.

sudo curl -s -o /etc/dnsmasq.d/adList.conf https://dl.dropboxusercontent.com/u/16366947/Files/adList-2015-02.conf

To address this performance issue, I have been working on a new branch of the project that uses /etc/hosts in place of /etc/dnsmasq.d/adList.conf . Thanks to some suggestions from you, I am pleased to say that it is working much faster, even with blocking 900,000+ ad domains (up from 120,000). It is also working over IPv6. It blocks Hulu ads (but doesn’t skip them–yet) on an Amazon Fire TV. Skipping Hulu ads is a big work in progress. For now, I’m just glad the Pi-hole is working!

Improvements (Coming Soon)

The Pi-hole will block ads from over 900,000 domains
Increased speed by using /etc/hosts instead of a dnsmasq config file
Increased speed by setting an expiration on queries
IPv6 support
A better script that makes it easier to add in additional sources to get domains from
and more…

The New Pi-hole In Action

I can now load up my Pi-hole and it reads the domains from /etc/hosts . When dnsmasq starts, the log file looks like this. Line 5 shows the /etc/hosts file being read.

Jun  2 11:09:42 raspberrypi ntpd[2275]: peers refreshed
Jun  2 11:09:42 raspberrypi ntpd[2275]: Listening on routing socket on fd #19 for interface updates
Jun  2 11:09:42 raspberrypi ntpd[2275]: restrict: error in address '::' on line 38. Ignoring...
Jun  2 11:09:42 raspberrypi ntpd[2275]: restrict: error in address '::1' on line 42. Ignoring...
Jun  3 23:09:22 raspberrypi dnsmasq[2126]: read /etc/hosts - 920565 addresses
Jun  3 23:12:15 raspberrypi dnsmasq[2126]: exiting on receipt of SIGTERM
Jun  3 23:12:17 raspberrypi dnsmasq[2451]: started, version 2.62 cachesize 10000
Jun  3 23:12:17 raspberrypi dnsmasq[2451]: compile time options: IPv6 GNU-getopt DBus i18n IDN DHCP DHCPv6 no-Lua TFTP conntrack
Jun  3 23:12:17 raspberrypi dnsmasq[2451]: using nameserver 8.8.4.4#53
Jun  3 23:12:17 raspberrypi dnsmasq[2451]: using nameserver 8.8.8.8#53

Also, once I got IPv6 is enabled, the IPv6 queries (AAAA) will also get redirected to the loopback. I decided to enable IPv6 because I found that some ads would failover to IPv6 if it couldn’t be reached on IPv4.

Jun  4 00:27:38 raspberrypi dnsmasq[3068]: query[A] googleads.g.doubleclick.net from 192.168.1.100
Jun  4 00:27:38 raspberrypi dnsmasq[3068]: /etc/hosts googleads.g.doubleclick.net is 127.0.0.1
Jun  4 00:27:38 raspberrypi dnsmasq[3068]: query[AAAA] googleads.g.doubleclick.net from 192.168.1.100
Jun  4 00:27:38 raspberrypi dnsmasq[3068]: /etc/hosts googleads.g.doubleclick.net is ::1
Jun  4 00:27:38 raspberrypi dnsmasq[3068]: query[A] rcm-na.amazon-adsystem.com from 192.168.1.100
Jun  4 00:27:38 raspberrypi dnsmasq[3068]: /etc/hosts rcm-na.amazon-adsystem.com is 127.0.0.1
Jun  4 00:27:38 raspberrypi dnsmasq[3068]: query[AAAA] rcm-na.amazon-adsystem.com from 192.168.1.100
Jun  4 00:27:38 raspberrypi dnsmasq[3068]: /etc/hosts rcm-na.amazon-adsystem.com is ::1

Testing The Ad Blocking Power Of The Pi-hole

I needed to find some better ad-filled sites to test how well the Pi-hole would block them. Below are a few of them (with some fun videos of the difference). Once you get your new Pi-hole working, you can use these links to test it out as well.

This site is way overdone with ads an other Internet clutter. Once blocked, you realize there is more ads than there is content.

Adsreview is one of my favorite ones to block because the site actually looks really nice once all the ads are gone.

Impressions

I have been using the new Pi-hole for about a week now and haven’t run into any problems. Next up, I will be re-creating it from scratch while writing up a blog post with instructions, exactly like I did for the original one.

Can’t Wait?

If you like to tinker around and want to try it out yourself right now, just replace your gravity.sh script with this version (be sure to delete or move your /etc/dnsmasq.d/adList.conf first so it doesn’t try to read it. You will also need to update your index.html , lighttpd.conf (to use IPv6 and the very tiny 25 Byte GIF) and dnsmasq.conf files, but all the files are available on Github as well for you to play with.

Appendix

Here is some of the sites being blocked (on IPv4 and IPv6) viewable with the command: cat /var/log/daemon.log | grep /etc/hosts

Jun  6 01:10:56 raspberrypi dnsmasq[2188]: /etc/hosts ads.flurry.com is ::1
Jun  6 01:10:56 raspberrypi dnsmasq[2188]: /etc/hosts ads.flurry.com is 127.0.0.1
Jun  6 01:12:03 raspberrypi dnsmasq[2188]: /etc/hosts fast.wistia.net is 127.0.0.1
Jun  6 01:12:03 raspberrypi dnsmasq[2188]: /etc/hosts fast.wistia.net is ::1
Jun  6 01:12:04 raspberrypi dnsmasq[2188]: /etc/hosts js.hs-analytics.net is 127.0.0.1
Jun  6 01:12:04 raspberrypi dnsmasq[2188]: /etc/hosts js.hs-analytics.net is ::1
Jun  6 01:12:05 raspberrypi dnsmasq[2188]: /etc/hosts www.snapengage.com is 127.0.0.1
Jun  6 01:12:05 raspberrypi dnsmasq[2188]: /etc/hosts www.snapengage.com is ::1
Jun  6 01:19:56 raspberrypi dnsmasq[2188]: /etc/hosts widget.perfectmarket.com is ::1
Jun  6 01:19:56 raspberrypi dnsmasq[2188]: /etc/hosts widget.perfectmarket.com is 127.0.0.1
Jun  6 01:19:56 raspberrypi dnsmasq[2188]: /etc/hosts z-na.amazon-adsystem.com is ::1
Jun  6 01:19:56 raspberrypi dnsmasq[2188]: /etc/hosts z-na.amazon-adsystem.com is 127.0.0.1
Jun  6 01:19:56 raspberrypi dnsmasq[2188]: /etc/hosts pagead2.googlesyndication.com is ::1
Jun  6 01:19:56 raspberrypi dnsmasq[2188]: /etc/hosts pagead2.googlesyndication.com is 127.0.0.1
Jun  6 01:19:56 raspberrypi dnsmasq[2188]: /etc/hosts wms.assoc-amazon.com is ::1
Jun  6 01:19:56 raspberrypi dnsmasq[2188]: /etc/hosts wms.assoc-amazon.com is 127.0.0.1
Jun  6 01:19:56 raspberrypi dnsmasq[2188]: /etc/hosts tags.bkrtx.com is ::1
Jun  6 01:19:56 raspberrypi dnsmasq[2188]: /etc/hosts tags.bkrtx.com is 127.0.0.1
Jun  6 01:19:57 raspberrypi dnsmasq[2188]: /etc/hosts www.googletagservices.com is ::1
Jun  6 01:19:57 raspberrypi dnsmasq[2188]: /etc/hosts www.googletagservices.com is 127.0.0.1
Jun  6 01:19:58 raspberrypi dnsmasq[2188]: /etc/hosts t.zqtk.net is ::1
Jun  6 01:19:58 raspberrypi dnsmasq[2188]: /etc/hosts t.zqtk.net is 127.0.0.1
Jun  6 01:19:58 raspberrypi dnsmasq[2188]: /etc/hosts cdn.krxd.net is ::1
Jun  6 01:19:58 raspberrypi dnsmasq[2188]: /etc/hosts cdn.krxd.net is 127.0.0.1
Jun  6 01:19:58 raspberrypi dnsmasq[2188]: /etc/hosts stats.g.doubleclick.net is ::1
Jun  6 01:19:58 raspberrypi dnsmasq[2188]: /etc/hosts stats.g.doubleclick.net is 127.0.0.1
Jun  6 01:19:58 raspberrypi dnsmasq[2188]: /etc/hosts secure-us.imrworldwide.com is ::1
Jun  6 01:19:58 raspberrypi dnsmasq[2188]: /etc/hosts secure-us.imrworldwide.com is 127.0.0.1
Jun  6 01:19:58 raspberrypi dnsmasq[2188]: /etc/hosts cdn.taboola.com is ::1
Jun  6 01:19:58 raspberrypi dnsmasq[2188]: /etc/hosts cdn.taboola.com is 127.0.0.1
Jun  6 01:19:59 raspberrypi dnsmasq[2188]: /etc/hosts p.typekit.net is ::1
Jun  6 01:19:59 raspberrypi dnsmasq[2188]: /etc/hosts p.typekit.net is 127.0.0.1
Jun  6 01:19:59 raspberrypi dnsmasq[2188]: /etc/hosts metrics.brightcove.com is ::1
Jun  6 01:19:59 raspberrypi dnsmasq[2188]: /etc/hosts metrics.brightcove.com is 127.0.0.1
Jun  6 01:19:59 raspberrypi dnsmasq[2188]: /etc/hosts pixel.staticworld.net is ::1
Jun  6 01:19:59 raspberrypi dnsmasq[2188]: /etc/hosts pixel.staticworld.net is 127.0.0.1
Jun  6 01:20:03 raspberrypi dnsmasq[2188]: /etc/hosts pubads.g.doubleclick.net is 127.0.0.1
Jun  6 01:20:03 raspberrypi dnsmasq[2188]: /etc/hosts pubads.g.doubleclick.net is ::1
Jun  6 01:20:04 raspberrypi dnsmasq[2188]: /etc/hosts imasdk.googleapis.com is ::1
Jun  6 01:20:04 raspberrypi dnsmasq[2188]: /etc/hosts imasdk.googleapis.com is 127.0.0.1
Jun  6 01:20:04 raspberrypi dnsmasq[2188]: /etc/hosts b.scorecardresearch.com is ::1
Jun  6 01:20:04 raspberrypi dnsmasq[2188]: /etc/hosts b.scorecardresearch.com is 127.0.0.1
Jun  6 01:20:05 raspberrypi dnsmasq[2188]: /etc/hosts static.chartbeat.com is ::1
Jun  6 01:20:05 raspberrypi dnsmasq[2188]: /etc/hosts static.chartbeat.com is 127.0.0.1
Jun  6 01:20:17 raspberrypi dnsmasq[2188]: /etc/hosts cdn.viglink.com is 127.0.0.1
Jun  6 01:20:17 raspberrypi dnsmasq[2188]: /etc/hosts cdn.viglink.com is ::1
Jun  6 01:20:18 raspberrypi dnsmasq[2188]: /etc/hosts www.googletagservices.com is 127.0.0.1
Jun  6 01:20:18 raspberrypi dnsmasq[2188]: /etc/hosts www.googletagservices.com is ::1
Jun  6 01:20:18 raspberrypi dnsmasq[2188]: /etc/hosts z-na.amazon-adsystem.com is 127.0.0.1
Jun  6 01:20:18 raspberrypi dnsmasq[2188]: /etc/hosts z-na.amazon-adsystem.com is ::1
Jun  6 01:20:18 raspberrypi dnsmasq[2188]: /etc/hosts cdn.taboola.com is 127.0.0.1
Jun  6 01:20:18 raspberrypi dnsmasq[2188]: /etc/hosts cdn.taboola.com is ::1
Jun  6 01:20:18 raspberrypi dnsmasq[2188]: /etc/hosts d8rk54i4mohrb.cloudfront.net is 127.0.0.1
Jun  6 01:20:18 raspberrypi dnsmasq[2188]: /etc/hosts d8rk54i4mohrb.cloudfront.net is ::1
Jun  6 01:20:18 raspberrypi dnsmasq[2188]: /etc/hosts widget.perfectmarket.com is 127.0.0.1
Jun  6 01:20:18 raspberrypi dnsmasq[2188]: /etc/hosts widget.perfectmarket.com is ::1
Jun  6 01:20:18 raspberrypi dnsmasq[2188]: /etc/hosts assets.pinterest.com is 127.0.0.1
Jun  6 01:20:18 raspberrypi dnsmasq[2188]: /etc/hosts assets.pinterest.com is ::1
Jun  6 01:20:18 raspberrypi dnsmasq[2188]: /etc/hosts static.parsely.com is 127.0.0.1
Jun  6 01:20:18 raspberrypi dnsmasq[2188]: /etc/hosts static.parsely.com is ::1
Jun  6 01:20:18 raspberrypi dnsmasq[2188]: /etc/hosts stats.g.doubleclick.net is 127.0.0.1
Jun  6 01:20:18 raspberrypi dnsmasq[2188]: /etc/hosts stats.g.doubleclick.net is ::1
Jun  6 01:20:18 raspberrypi dnsmasq[2188]: /etc/hosts cdn.krxd.net is 127.0.0.1
Jun  6 01:20:18 raspberrypi dnsmasq[2188]: /etc/hosts cdn.krxd.net is ::1
Jun  6 01:20:18 raspberrypi dnsmasq[2188]: /etc/hosts wms.assoc-amazon.com is 127.0.0.1
Jun  6 01:20:18 raspberrypi dnsmasq[2188]: /etc/hosts wms.assoc-amazon.com is ::1
Jun  6 01:20:18 raspberrypi dnsmasq[2188]: /etc/hosts pagead2.googlesyndication.com is 127.0.0.1
Jun  6 01:20:18 raspberrypi dnsmasq[2188]: /etc/hosts pagead2.googlesyndication.com is ::1
Jun  6 01:20:18 raspberrypi dnsmasq[2188]: /etc/hosts tags.bkrtx.com is 127.0.0.1
Jun  6 01:20:18 raspberrypi dnsmasq[2188]: /etc/hosts tags.bkrtx.com is ::1
Jun  6 01:20:18 raspberrypi dnsmasq[2188]: /etc/hosts t.zqtk.net is 127.0.0.1
Jun  6 01:20:18 raspberrypi dnsmasq[2188]: /etc/hosts t.zqtk.net is ::1
Jun  6 01:20:22 raspberrypi dnsmasq[2188]: /etc/hosts ads.flurry.com is ::1
Jun  6 01:20:22 raspberrypi dnsmasq[2188]: /etc/hosts ads.flurry.com is 127.0.0.1
Jun  6 01:20:26 raspberrypi dnsmasq[2188]: /etc/hosts metrics.brightcove.com is ::1
Jun  6 01:20:26 raspberrypi dnsmasq[2188]: /etc/hosts metrics.brightcove.com is 127.0.0.1
Jun  6 01:23:08 raspberrypi dnsmasq[2188]: /etc/hosts cdn.krxd.net is ::1
Jun  6 01:23:08 raspberrypi dnsmasq[2188]: /etc/hosts cdn.krxd.net is 127.0.0.1
Jun  6 01:23:09 raspberrypi dnsmasq[2188]: /etc/hosts www.googletagservices.com is ::1
Jun  6 01:23:09 raspberrypi dnsmasq[2188]: /etc/hosts www.googletagservices.com is 127.0.0.1
Jun  6 01:23:09 raspberrypi dnsmasq[2188]: /etc/hosts i.po.st is ::1
Jun  6 01:23:09 raspberrypi dnsmasq[2188]: /etc/hosts i.po.st is 127.0.0.1
Jun  6 01:23:09 raspberrypi dnsmasq[2188]: /etc/hosts www.google-analytics.com is ::1
Jun  6 01:23:09 raspberrypi dnsmasq[2188]: /etc/hosts www.google-analytics.com is 127.0.0.1
Jun  6 01:23:09 raspberrypi dnsmasq[2188]: /etc/hosts edge.quantserve.com is ::1
Jun  6 01:23:09 raspberrypi dnsmasq[2188]: /etc/hosts edge.quantserve.com is 127.0.0.1
Jun  6 01:23:09 raspberrypi dnsmasq[2188]: /etc/hosts secure-us.imrworldwide.com is ::1
Jun  6 01:23:09 raspberrypi dnsmasq[2188]: /etc/hosts secure-us.imrworldwide.com is 127.0.0.1
Jun  6 01:23:09 raspberrypi dnsmasq[2188]: /etc/hosts b.scorecardresearch.com is ::1
Jun  6 01:23:09 raspberrypi dnsmasq[2188]: /etc/hosts b.scorecardresearch.com is 127.0.0.1
Jun  6 01:23:09 raspberrypi dnsmasq[2188]: /etc/hosts widgets.outbrain.com is ::1
Jun  6 01:23:09 raspberrypi dnsmasq[2188]: /etc/hosts widgets.outbrain.com is 127.0.0.1
Jun  6 01:23:09 raspberrypi dnsmasq[2188]: /etc/hosts a.postrelease.com is ::1
Jun  6 01:23:09 raspberrypi dnsmasq[2188]: /etc/hosts a.postrelease.com is 127.0.0.1
Jun  6 01:23:09 raspberrypi dnsmasq[2188]: /etc/hosts mtrx.go.sonobi.com is ::1
Jun  6 01:23:09 raspberrypi dnsmasq[2188]: /etc/hosts mtrx.go.sonobi.com is 127.0.0.1
Jun  6 01:23:10 raspberrypi dnsmasq[2188]: /etc/hosts bonniercorp.122.2o7.net is ::1
Jun  6 01:23:10 raspberrypi dnsmasq[2188]: /etc/hosts bonniercorp.122.2o7.net is 127.0.0.1
Jun  6 01:23:11 raspberrypi dnsmasq[2188]: /etc/hosts static.chartbeat.com is ::1
Jun  6 01:23:11 raspberrypi dnsmasq[2188]: /etc/hosts static.chartbeat.com is 127.0.0.1

Let me know what you think in the comments below.