Block Millions Of Ads Network-wide With A Raspberry Pi-hole 2.0

pihole-millions

Update 2016-04-24 : Please note this guide is now out-date, and it is recommended that you use the automated install to install and update Pi-hole.  

Technically, this guide will still work, but it does not include any of the features and improvements that have been added.  This article will, however, still give you some insight into how the Pi-hole functions.

The Pi-hole can block ads for all devices on your network.  All you need is a Raspberry Pi connected to your router.  It was inspired as a low-cost, open source alternative to the AdTrap.  The Pi-hole works on the the BB+ and Pi 2 (it can also run on the Zero, but you need a micro-USB-to-Ethernet adapter)

Block Over 900,000 1.6 Million Ad-serving Domains (And More)

Currently, over 900,000 1.6 million (if you decide to use the mahakala source list) domains known to serve advertisements can be blocked before they even reach your computer.  The list of these domains are maintained by different sites across the Internet.   The Pi-hole queries these sites and compiles them into an aggregated list, which is then stored on the Raspberry Pi to block advertisements.

vegeta-pihole-over900000

Anytime a Website wants to load something from a domain on the list, it is redirected back to the Pi, so the real server is never contacted.  This can not only speed up your Web browsing, but it will also reduce your bandwidth usage because your browser is not downloading the ad from the server.

Setting It Up (The Easy Way)

These are the steps you will take to activate your Pi-hole and begin blocking ads.  If you prefer to tinker around and do things on your own, take a look at the manual setup.

  1. Install Raspbian onto the Pi
  2. Run this command: curl -L install.pi-hole.net | bash
  3. Configure your computers/devices to use the Pi as their DNS server [critical step]

Pi-hole Is Free, But Powered By Your Donations

That Magical Moment…

Once the installation is complete and you have configured your devices to use the Pi-hole as their DNS server, ads are instantly blocked without any configuration.  It’s one of those things that is difficult to describe and more something you need to experience.

Optional Settings After The Pi-hole Is Setup

Alternative Installation Method (DietPi Image)

If you want something even easier, you can try burning this DietPi image to your SD card.  It is already set up as a Pi-hole, but isn’t as up-to-date as the method above, but it is something we are working on.

If You Already Had A Pi-hole Setup…

It probably wasn’t working that well due to the last list update (unless you fixed it), so I suggest just getting a new SD card or wiping it and starting over.  Make a backup of it first if you are worried.

You will be better off installing the new Pi-hole (using the install command).  The original one is very primitive compared to all the enhancements that have been made.

Caveats

Some ads still get through via IPv6 (but there are some things you can try  to improve this) and https.  Near the end of 2015, I noticed a lot of the bigger ad networks started delivering a lot more ads via IPv6.  I am still working on improving IPv6 blocking, but as you may know the Pi-hole is free, but powered by your donations.

How It Works

First, I’ll explain the concepts.  Then, I’ll break down the how the script works.  After that, I’ll give some tips on customizing your Pi-hole to add more domains to block, or use alternative DNS servers.  Finally, I’ll cover, in detail, how to manually set this up.

The Concept

The Pi-hole answers your computer’s DNS queries and if it is a domain on the ad list, it sends the request to the Web server that is also installed on the Pi.  This Web server just serves up a blank page.  So instead of loading the ad from the real domain, the blank page from the Pi is downloaded, thus “blocking” the ad.

Understanding The Components

DNS

When you visit a Website by typing in the domain name (google.com), there is a server out on the Internet that looks up that domain’s IP address and sends you to the correct server (computers understand numbers, not letters). If you knew Google’s IP address, you could go directly there by typing that in instead of google.com.

Armed with the knowledge that each domain exists at a certain IP address, when you visit a Website, that site might be trying to load stuff from other domains (such as advertisements).  So your computer makes a DNS query to find out the IP address of that resource.  If you are using a DNS server on the Internet, it will download the correct content.  But since the Pi-hole is a DNS server, it can tell your computer that SomeAdDomain.com exists at a fake IP address.

Web Server

Now that we can tell your computer to download something from your fake IP address, we can use a Web server to serve up a tiny blank image file or Webpage.  So as far as your computer is concerned, it asked where a resource was, found it, and downloaded it.  From your point of view, you downloaded a blank file instead of the advertisement.

The hosts File

/etc/hosts  works just like DNS except that it’s faster because it is the first place your computer checks where a domain is.  It also can be used to do other things.  In the case of the Pi-hole, this file contains all the entries for the ad-serving domains, which tells your computer that the site exists at the IP address of the Rpi.

If you know about the hosts file, you may be thinking, why do you need a DNS server if the hosts file already does the redirecting?  And doesn’t the hosts file only apply to the computer it exists on?  Normally, yes, but when you tell dnsmasq  to use file, it give the hosts file a super power and makes the file available to any device that queries the DNS server (i.e. any device on your network).  In essence, transforming it into a DNS zone file.

The Script

After November 15, 2015, the script’s code was refactored, so this post is a little outdated from what you will find on Github, but it is still very relevant to understanding how everything works.  All the same concepts are still in the script–they have just been turned into functions to make the code easier to maintain.

Now that you know how the Pi-hole works, I’ll explain how the script was designed to find ad-serving domains and compile them into a huge list, which is then used to create the hosts file. Throughout the script, you will notice a science/Star Trek theme. So things like variable names and echo commands are references to that. 🖖

Get The Raspberry Pi’s IP Address

In order for clients to access the Pi’s Webserver, they need to know where to find it.  Make sure you set a static IP address as it will be stored in this variable and be used to populate the hosts file.

piholeIP=$(ip -4 addr show | awk '{match($0,/[0-9]+\.[0-9]+\.[0-9]+\.[0-9]+/); ip = substr($0,RSTART,RLENGTH); print ip}' | sed '/^\s*$/d' | grep -v "127.0.0.1")

Set Some Variables

So I don’t repeat myself (my golden rule of scripting), I use variables to store all of the filenames.

Getting The Lists (And Being A Respectful Netizen)

There are many sites out there that maintain lists of ad-serving domains.  These URLs are stored in the sources  array.  An array is used so we can loop through every domain inside and do something to it.  This makes it easier to add additional URLs whenever you want.

A for  loop is used to run some commands on every entry in the array.  First, I store the URL in a variable for better readability.

url=${sources[$i]}

Next, I extract just the domain name and store it in a variable, again, for better readability.

domain=$(echo "${sources[$i]}" | cut -d'/' -f3)

I also set the user agent in a variable as it seems to be a universal one that works for every URL that is being downloaded.

agent="Mozilla/10.0"

I create another variable to save the file with a consistent naming convention of list.indexNumber.domainName.  This helps with debugging and also names the files nicely in case you ever want to look at them.

saveLocation=$origin/"list"."$i"."$domain".$justDomainsExtension

Next, a case  statement determines if the the URL needs a special curl  command.  If it doesn’t, it sets another variable to simply hold the curl command.

cmd="curl"

Now, we are ready to download the lists. Previously, I was storing the lists in RAM (/run/shm) so it was faster, but that ended up causing a lot of errors when the list grew very large.  So now, the lists are downloaded directly to the SD card–but it doesn’t do it recklessly. Thanks to some bash-fu, we can reduce the amount of I/O that takes place while also respecting the bandwidth of the sites that are hosting the lists.

First, a temporary file is made.

patternBuffer=$(mktemp)

Then it checks if the list from the source already exists and is (-r)eadable. If it isn’t, another variable is set, which contains another part of the curl command.

if [[ -r $saveLocation ]]; then
heisenbergCompensator="-z $saveLocation"
fi

Now that all the components of the curl command have been established, we can execute it to downloaded the list.

$cmd -s $heisenbergCompensator -A "$agent" $url > $patternBuffer

If there is (-s)omething in the $patternBuffer, then the list can be transported to your computer.

The content in the $patternBuffer is in a state of flux and needs to be materialized. Lieutenants sed and awk take care of this and save the cleaned up list into the $saveLocation declared earlier.

awk '($1 !~ /^#/) { if (NF>1) {print $2} else {print $1}}' $patternBuffer | sed -nr -e 's/\.{2,}/./g' -e '/\./p' > $saveLocation

If there are no changes, the list is skipped as there is no need to re-download it. Doing it this way respects the bandwidth of the people operating the sites as there is no need to download the entire list if the exact same one is already on the computer.  Some of the sites will block your IP address if you try to download from them too often, which I learned the hard way while trying to test out the Pi-hole.

Another benefit of downloading it locally is that it you always have a copy if the site is no longer available.

Now, while many of the lists downloaded are already in hosts format, their formatting is inconsistent and I like things to look nice and readable, so I have some code to format everything the same way.  For example, one list has a single space between the IP address and the domain name, while others have tabs or additional whitespace.

So one list looks like this:

And another like this:

After the sed and awk finish processing the file, only the domain name remains, allowing me to consistently format them in another step.

Aggregating The Lists

Now that there are files from each source that contain only domain names, it is time to aggregate them into a single list while removing any carriage returns or line feeds.

find $origin/ -type f -name "*.$justDomainsExtension" -exec cat {} \; | tr -d '\r' > $origin/$matter

Blacklist entries are appended to the list if /etc/pihole/blacklist.txt  exists.

cat $blacklist >> /tmp/matter.txt

The /etc/pihole/whitelist.txt  file is also parsed if it exists and is stored in a temporary file because it needs special formatting to properly find the domains to remove.

awk -F '[# \t]' 'NF>0&&$1!="" {print "^"$1"$"}' $whitelist | sed 's/\./\\./g' > $latentWhitelist

In order for grep  to accurately remove entries from the list, a dollar sign ($) must be appended to the end of each line.  Then, you can use the -x option to remove the entries.

grep -vxf $latentWhitelist $origin/$matter > $origin/$andLight

Also, since we want to be able to access our sources for future updates, those get whitelisted as well in case they happen to end up on one of the lists.

Formatting The List

Now that there is a full list of domains with the blacklist and whitelist applied, I use sed to remove some different types of line feeds (invisible characters at the end of a line of text), which if not removed, result in duplicate domains in the list that sort cannot detect.

cat $origin/$andLight | sed $'s/\r$//' | sed '/^\s*$/d' > $origin/$supernova

Finally, the list can be alphabetized and duplicates can be removed.

sort -u $origin/$supernova > $origin/$eventHorizon

The list is still just a list of domains names, so it needs to be formatted like a hosts file with the IP address, a space, and then the domain name.

cat $origin/$eventHorizon | awk '{sub(/\r$/,""); print "'"$piholeIP"' " $0}' > $origin/$accretionDisc

The last step is to copy the formatted file over as /etc/pihole/gravity.list.

sudo cp $origin/$accretionDisc $adList

If your brain isn’t hurting too much after reading that, you can continue on to learn how to block malware and tracking sites by adding our own sources to the script and making the hosts file even more powerful.

Blocking Malware, Tracking Sites, And More By Adding Your Own Sources

Recently, I rebuilt the gravity.sh  script (the magic behind the Pi-hole) from the ground up to make it easier to add your own sources.  You will need to know how to write shell scripts to do this (but you might be able to figure it out by reading through this section).  This will be done by editing /usr/local/bin/gravity.sh  on your Raspberry Pi-hole.

Add Sources That Have A List Of Domains

Before actually editing the script, you should curl  the URL of the site that has the list of domains you want to block and figure out how to get just the domain names from it.  If the list you are downloading is already in hosts format (IP address, whitespace, domain name), then you only need to add your domain(s) into the sources array by putting it in single quotes–one domain per line–and the script will take care of the rest.

If your list needs additional formatting to get just the domain name, you still need to put it in the sources arrary, but then you need to figure out how to parse out only the domain names (sed and awk will be your friends).

As an example, this list blocks spam domains, but it has a lot of characters to remove like exclamation points and carats.  So the following sample from the list found at http://spam404bl.com/spam404scamlist.txt

would need to be parsed down to:

Once you figured out the command to use, you will need to add an entry inside the case statement.  Put the domain name in quotes followed by a parenthesis.  After the parenthesis, put your curl command in but follow the same format as the other entries already in the statement.

Now, just execute the gravity.sh  script again and it will download and append the new list!

Additional DNS Blacklists

Here are a few links to get you started.  I haven’t verified if any of these lists are accurate or still updated.  They just came up in my search while I was creating the Pi-hole.  So add them to script at your own risk.

  • http://cdn.files.trjlive.com/hosts/hosts-v8.txt
  • https://s3.amazonaws.com/lists.disconnect.me/simple_tracking.txt
  • https://s3.amazonaws.com/lists.disconnect.me/simple_ad.txt
  • https://zeustracker.abuse.ch/blocklist.php?download=domainblocklist
  • http://sysctl.org/cameleon/hosts
  • http://securemecca.com/Downloads/hosts.txt
  • https://hosts.neocities.org/
  • http://tcpdiag.dl.sourceforge.net/project/adzhosts/HOSTS.txt
  • http://optimate.dl.sourceforge.net/project/adzhosts/HOSTS.txt
  • http://mirror1.malwaredomains.com/files/justdomains
  • https://raw.githubusercontent.com/reek/anti-adblock-killer/master/anti-adblock-killer-filters.txt
  • http://spam404bl.com/spam404scamlist.txt
  • http://malwaredomains.lehigh.edu/files/domains.txt
  • http://download.networktoolbox.de/dnsmasq.conf.tail
  • http://jansal.googlecode.com/svn/trunk/adblock/hosts
  • http://www.sa-blacklist.stearns.org/sa-blacklist/sa-blacklist.current
  • https://easylist-downloads.adblockplus.org/malwaredomains_full.txt
  • https://easylist-downloads.adblockplus.org/easyprivacy.txt
  • https://easylist-downloads.adblockplus.org/easylist.txt
  • https://easylist-downloads.adblockplus.org/fanboy-annoyance.txt
  • http://www.fanboy.co.nz/adblock/opera/urlfilter.ini
  • http://www.fanboy.co.nz/adblock/fanboy-tracking.txt

Using Different DNS Servers

You can change the secondary and tertiary DNS servers to use.  In the automated setup, I just use Google’s public ones, but you might enjoy using OpenDNS, which offers more features.  They are found at 208.67.222.222 and 208.67.220.220.  These settings should be changed in /etc/dnsmasq.conf as shown below.

You can also use any DNS server of your choosing.  There might be a faster one based on your geographical location.


 

Setting It Up Manually (Building It From Scratch)

This section will give a detailed walkthrough of setting up a Pi-hole from scratch.  You can also reference the original article for more information but it might confuse you more since it is pretty primitive and outdated compared to this article.

Conceptual Overview

  1. Install Raspbian
  2. Set a static IP address
  3. Install the Web server
  4. Install the DNS server
  5. Configure DNS to use secondary and tertiary public DNS servers to look up legitimate sites
  6. Configure DNS to use the hosts file as the first location to query (so ad queries can be blocked)
  7. Configure the Web server to redirect ad domains back to the Pi
  8. Create a blank HTML page or a tiny GIF file to take the place of the ads
  9. Create a script to download the lists and save it has the hosts file
  10. (optional) Install a streaming server to attempt to skip video/audio ads (not just block them)
  11. Set your computer to use the Pi as its DNS server [Critical Step]
  12. Test it out!

Install Raspbian

There are plenty of guides out there on how to do this, so I won’t cover it here.  But you should update everything once it is installed using these commands:

sudo apt-get -y update
sudo apt-get -y upgrade

Set A Static IP Address

This is the file that configures how your Raspberry Pi connects to the network.  There are two ways to set the IP address.  The first is via DHCP, in which the IP address is automatically assigned–this is what your Pi will have set by default.  Second, is by using a static address, which is useful if you want to always know what address to find the Pi at (although a similar effect can be achieved via a DHCP reservation).

A static IP address will work better for this project, so I will show that configuration first.  This is because your Pi is acting as a server, so when you put entries into the hosts file with the Pi’s IP address, that address should not change, which is a possibility when using DHCP.

Static Configuration

You will need to know a little bit about your network including the subnet mask, gateway (IP of the router), and network address to set a static IP.  Below is a sample config that may work for most people, but you may need to adjust it depending on how your network is setup.

Your network addressing may begin with 192.168.0.x and not 192.168.1.x.  You may even be using a 172.16.0.0 or 10.x.x.x address.  But since I made this project to be used at home, your router most likely has the 192.168.x.x address.  Either way, you will need to modify the file to suit your environment.

The method to set a static address is different in Raspbian, “Jessie.”  You should now use /etc/dhcpcd.conf to do this.  

Set A Static IP In Raspbian Jessie

sudo vi /etc/dhcpcd.conf

Append this to the bottom of the file, substituting the correct IP address you want.

Set A Static IP In Raspbian Wheezy

Modify/add the highlighted lines in so the file looks like below. (the config below only works for “Wheezy.”  See above for the latest method to set a static IP.

sudo vi /etc/network/interfaces

DHCP Configuration

DHCP won’t work well for the project.  If your IP address happens to change (because it is not set statically) you wouldn’t be able to properly access Web pages and you would have to run the gravity.sh script again to re-create the hosts file with the new IP address.  While it is possible to use a DHCP reservation to always get the same IP, it’s usually better to just set a static IP address instead.

If you do decide to use DHCP, you do not need to change anything as the Pi already has this set.  But you will need to run the script anytime your IP address changes.

Install The Web Server

It is possible to use the Pi-hole without a Web server.  Requests will just return an error instead of a Webpage because nothing will exist at the loopback address.

Speed-wise, choosing to use a Web server or not is negligible.  But using the Web server gives you a lot more functionality including the ability to improve performance with the use of mod_expire , redirect stubborn types of ads that aren’t blocked just by domain name, and the ability to send out your own content in place of ads.  You also get some great logging features, which can be used for some fun stuff.

Install lighttpd

sudo apt-get -y install lighttpd

Adjust Permissions

Make www-data  the new owner of the directory that serves up the Webpage:

sudo chown www-data:www-data /var/www/html

Modify the permissions to make it writeable:

sudo chmod 775 /var/www/html

Add the pi user to the www-data  group:

sudo usermod -a -G www-data pi

Install The DNS server

sudo apt-get -y install dnsmasq

Once this finishes, all of the necessary components are installed, but they still need to be configured.

Start DNS When The System Boots

dnsmasq  won’t start at boot by default, so you need to tell it to do so.

sudo update-rc.d dnsmasq enable

Edit The DNS Config File

/etc/dnsmasq.conf  is the file that contains all of the settings for dnsmasq , so that is the file you need to edit.  Linux config files are nothing more than simple text files with specific options that a program reads.  So all you need to do is put those options in the file.

First, rename to original file so it’s available as a backup in case ever you want a clean version of the file.

sudo mv /etc/dnsmasq.conf /etc/dnsmasq.orig

Then create a new /etc/dnsmasq.conf as shown below:

sudo vi /etc/dnsmasq.conf

Take note of the server.  These are the secondary and tertiary DNS servers to use for lookups if the first DNS server (your Raspberry Pi) doesn’t have a record for the site.  These are necessary to look up legitimate Websites because when we are done, your Pi’s DNS service will only know about the ad domains you put in your hosts file; it won’t know about any real Webpages on the Internet unless you have those values set.

Configure DNS To Use Secondary And Tertiary Public DNS Servers To Look Up Legitimate Sites

The config above uses Google’s public DNS servers (8.8.8.8 and 8.8.4.4) but you may want to consider using OpenDNS, which might be a bit faster and also provides some other features such as parental controls and filtering.  They are found at 208.67.222.222 and 208.67.220.220.  To use them, just put those IP addresses in place of Google’s.

The options set in the config file above are as follows:

  • domain-needed:  never forward names without a dot or domain part.  This prevents single hostnames–like raspberrypi –from being forwarded to the upstream DNS servers.  There is no need to ask the Internet where that computer is, because it only exists on your local network.
  • bogus-priv: never forward reverse-lookup queries which have the local subnet’s IP range to upstream.  Your local network, probably 192.168.x.x, doesn’t exist on the Internet, so similar to domain-needed, these do not need to be forwarded upstream.
  • no-resolv: do not use /etc/resolv.conf  (see below–this method is a bit easier than using a separate file).  That file can get overwritten when using DHCP, so I use this in the Pi-hole by simply adding the server option shown below
  • server: upstream name servers to use.  Your Pi already uses itself, so you just need to put two public DNS servers here
  • interface: specifies that the ethernet port is used
  • listen-address: this tells what address to listen for queries on.  If using IPv6, you will need another entry that listens on ::1
  • cache-size: set as large as possible (10,000) to allow for super-fast DNS queries
  • log-queries: logs all name resolutions to /var/log/daemon.log (the default location–see below), which allows you to see what sites are being visited, which can also be viewed in real time while your Pi-hole is running.
  • log-facility=/var/log/pihole.log: chooses a different location for the logs to go to.  Since all DNS queries will pass through the Pi-hole, it should have it’s own log file (daemon.log is a shared log between any running daemons).  This make it easier to parse through and extract data.
  • log-async: this allows it to continue functioning without being blocked by syslog, and allows syslog to use dnsmasq for DNS queries without risking deadlock
The Resolver File (Not Used In The Pi-hole, But You Can If You Want)

/etc/resolv.conf  is a file that sets what servers to use to try to resolve domain names (this file does the same thing as the server= value we set in /etc/dnsmasq.conf).  dnsmasq  checks this file by default unless you use the no-resolv  option, which we already did above.  You can choose to use this file, but it is easier to have everything in one place.

Also, this file will cause problems if you are using DHCP (another reason to use a static IP).  Unless you have your DHCP server (usually the router) set to use the Pi-hole as its primary DNS, it will overwrite the file on the next reboot, wiping out your custom settings.

There can only be three entries in this file for name servers.  Add Google’s public DNS servers, OpenDNS ones, or others of your choosing.

nameserver 8.8.8.8
nameserver 8.8.4.4

As previously mentioned, the Pi will check with itself first, so the entries above will allow all queries to go through the Raspberry Pi first, and if it cannot resolve the name, it will use the servers you set.

Configure DNS To Use The Hosts File As The First Location To Query

For simplicity, I designed the Pi-hole to use /etc/pihole/gravity.list, which is just a file formatted like /etc/hosts  but lives in a different location (I still call it a “hosts” file).  It works fast with over 900,000 entries.

You actually do not need to do anything to configure dnsmasq  to read the real hosts file (/etc/hosts) first; your computer already does that by default.  By simply adding entries into the hosts file, it will just start working.  But there is actually another reason I included this section in the article.

There is a special option you can enable in the dnsmasq  config file, /etc/dnsmasq.conf , which allows you to have a file formatted as the hosts file, but it can exist anywhere on the system and be named anything.  If this an option you want to use, your dnsmasq.conf  file should contain a line like this:

addn-hosts=/path/to/hostsfile

So you can choose to use the option above if you want the file to exist somewhere else and leave the hosts file alone.

Configure The Web Server To Redirect Ad Domains To A Blank Page

Now that DNS is able to redirect advertisement queries back to itself using the hosts file, we can use the Web server to point them to a blank Webpage (or other resources like a tiny transparent GIF file).

Just like dnsmasq , the Web server has a text config file: /etc/lighttpd/lighttpd.conf .  Backup the original file in the same fashion as the dnsamsq  config file.

sudo mv /etc/lighttpd/lighttpd.conf /etc/lighttpd/lighttpd.orig

Create a new file as shown below.

sudo vi /etc/lighttpd/lighttpd.conf

Then, restart lighttpd  to apply everything.

sudo service lighttpd restart

Understanding The lighttpd Config File

This file is way more complex than the dnsmasq  one, so I will break each part down and explain what it does.  This should also help if you want to modify the file to redirect the requests somewhere else (i.e. a streaming server), which is what you probably need to do to block video ads like those that Hulu Plus has (skipping video ads is still a work in progress).

First, the additional modules are loaded.  This is like turning on a feature so you can use it.

Then, there are a bunch of server settings, like the port of the server and the folder where the Website resources are stored.

These are some other basic settings, which I honestly don’t know if you need them or not.  But all the resources I researched had these in them, so I just did the same.

The next lines say to use IPv6, and use some config files.

Now we get into the custom settings that are specific to the Pi-hole.  This first part says that if the URL points to something in the /var/www/pihole  folder, it’s expiration time should be one day.

This is one of the ways the Pi-hole saves processing; the client doesn’t need to contact the Pi-hole about this URL for one day.  So when a query comes through, it doesn’t have to see if there is something new on the page.  And really, this could be set to any time frame because it’s all pointing to your fake Web server anyway.

Finally, this next line says that every domain should be sent to /var/www/pihole/index.html .  Since the only requests that will get routed back to the Pi are ad domains, we can use the star (*) as a wildcard to send it all into the “black hole”.

You can use rewrite or redirect here.  The difference? url.rewrite  doesn’t change the URL client-side.  So if you went to doubleclick.net, it would show you went to doubleclick.net even though you really were being shown a page from /var/www/pihole/ .  url.redirect  still sends you somewhere else, but you would see the change on your end.  It’s kind of confusing, but this link explains the differences better than I can.

Create A Blank HTML Page Or A Tiny GIF File To Take The Place Of The Ads

The Web server now knows to send ad domains to /var/www/pihole/index.html , but that file doesn’t exist yet.  There are a few options you can choose from for this Webpage.  Some work better than others, so experiment with what works best for you.  I have found that a simple blank Webpage works the best in most situations, but you can really make it into whatever you want:

sudo mkdir /var/www/pihole
sudo vi /var/www/pihole/index.html

There are a few other options you can try.  One is to just show a tiny 1×1 transparent image file (it should be save in /var/www/html/pihole ).  That image file can be replaced with a GIF of different sizes, with the smallest being 25 Bytes.

That seemed to work OK for me, but I found it worked better if I put it in a <div></div> and defined the height and width:

Another fun option is to use a base64 encoded 1×1 GIF.  The cool part is that you do not even need an actual file to do this.  I didn’t have as much luck with this, but it’s still pretty cool.  Just create the index.html  page as shown below:

One last option is to not use an .html file at all and instead, rewrite/redirect the pages directly to the image file by setting it like this in /etc/lighttpd/lighttpd.conf :

It’s impossible to test every site, so mess around and choose the one that seems to work best for you.  As I mentioned previously, the simple blank page worked the best for me.

Create A Script To Download The Lists And Save It As The Hosts File

Download the script (reminder: the script found at the link is much different now, but the concepts are still the same) and make sure it is executable.

curl -o /usr/local/bin/gravity.sh https://raw.githubusercontent.com/jacobsalmela/pi-hole/master/gravity.sh
chmod 755 /usr/local/bin/gravity.sh

Now that everything is set up, you need to actually get the list of the domains by running the script (make sure your static IP address is set, or your hosts file will be misconfigured).

sudo /usr/local/bin/gravity.sh

I covered the script in detail earlier, so I will just cover a few lines you may want to change depending on your setup.  If you do edit it, you will just need to run the script again.

Changing Variables

First off, you may want to change the location of some of the files generated by the Pi-hole.  This will save the files somewhere else.  Leave off the trailing slash of directories.  Also, if you decided to use the addn-hosts=  option in dnsmasq.conf , make sure that that path matches what you put in the $adList  variable below.

origin=~/etc/pihole
adList=/etc/hosts

(Optional) Install A Streaming Server To Attempt To Block Video/audio Ads

I will be covering this in a separate article, because this article is already long enough!  I will update it when I get better results.  For now, experiment with the link in the header of this section or check out this pull request.

Set Your Computer To Use The Pi As Its DNS Server [Critical Step]

If you don’t set your computer to use the Pi as it’s DNS server, you won’t be able to benefit from all of your hard work.  This is different for every operating system, but Google’s instructions are pretty helpful.

Set The DNS Server on Your Router Instead Of Configuring Each Device On Your Network

It is possible to have your router force your network devices to use the Pi-hole as their DNS server.  It involves changing the DNS server in the DHCP options section of your router.  To work properly, it should be the only DNS server you set, even though there will likely be fields for two more (don’t forget, the Pi-hole is already configured to use other upstream servers).

Unfortunately, this part is a bit difficult to describe as every router is different.  Also be aware that it might not work with some routers.

Test Functionality

Restart the Pi after making all these changes.  I have an FAQ setup for testing and troubleshooting, but I also explain some of the tests you can run below.

sudo shutdown -r now

Test One: Check DNS And The Hosts File

From a computer that is using the Pi as its DNS server, ping an ad domain and it should return the IP address of your Rpi (not the real IP of the domain).

ping 247media.com

If you get replies, you know dnsmasq  and the hosts file and dnsmasq  are both working properly

Test Two: Check The Web Server

As long as your computer is on the same network as the Pi, you can view the blank Webpage by navigating to the Pi’s IP address from a browser.  If you are in Chrome or Firefox, you can right click > Inspect Element and verify that it is your HTML code.

lighttpdworking

Test Three: Check Name Resolution And The Web Server Together

Try going to a domain on the ad list in a browser.  If it loads your Pi’s Webpage, then both services are working correctly.  When originally setting this up, I had the hosts file on the RPi using 127.0.0.1, but this meant that when a client requested the domain, it would try to connect to its loopback address, not the Pi’s.  The Webpage would not load because the Web server wasn’t running on the client (127.0.0.1).  This also explained why I could access the Web page via IP address but not via domain name.

Test 4: Verify Upstream DNS

And Observe The Speed And Performance Benefits Of The Pi-hole

You can verify that the Pi-hole is actually making your DNS queries–thus browsing the Internet–faster.  Make sure you have your computer using the Pi as its DNS server and then run the dig  command on a domain you have never visited before.  It doesn’t need to be an advertising domain; it can be any domain as long as you haven’t been to it yet (this will ensure the response has not already been cached).

In the example below, I dig a site that I have not been to and is not an ad server.

dig pi-hole.net

Take note of two things in the output of the command below:

  1. the Website’s real address is shown
  2. the response time is 148ms.

Now, take a look at the results when I run the same command again.  Thanks to the caching we enabled, the response time is now 6ms; much better!  This happens because we set the option cache=10000  in the /etc/dnsmasq.conf  file.  This cache is stored in RAM and will be flushed whenever the service is restarted.

Next, I dig  a site that is an ad server to show that it get’s redirected back to the Pi and not the real IP address of the domain.

dig a-ads.com

And again, when I run it the second time (see the faster response time):


Create A Recurring Task to Update the Ad List

If you installed the Pi-hole using the automated install command, a system-wide crontab is installed to /etc/crontab, which will run do this for you, but you can also make your own as described below.

In case any the Websitew that compile the lists of ad servers ever updates, set gravity.sh  to run once a week using crontab .  This is much more respectful to do now than the original Pi-hole was because this script will only download the list from the site if there are changes.

sudo crontab -e

Append the following line to the bottom of the file and save it:

@weekly /usr/local/bin/gravity.sh

Now the ad list should always stay up-to-date if the Website it is being pulled from ever updates it.  You can just leave your Pi-hole plugged in next to your router and always be blocking the most ads possible!


Alternative To /etc/hosts

You don’t have to use the hosts file (or addn-hosts ), but performance starts to suffer once the list of domains gets past 120,000.  Instead, you can use the /etc/dnsmasq.d  folder.  Any file in this directory is considered a config file and is loaded up when dnsmasq  starts.  The syntax of the example line below says “if the address queried is 101com.com (an ad server) send it to this IP address (the Pi–192.168.1.117 or whatever you set it to).”

address=/101com.com/192.168.1.117

This file is full of these rules, which are loaded when dnsmasq  is.  If a DNS query matches one of the rules, it will send the request to 192.168.1.117, which is the Raspberry Pi.  Since the request gets redirected there, it won’t reach your other device(s).

One problem with this method is that the file is a lot bigger than a hosts file because of the additional characters.  Because of this and the sheer quantity of domains to parse through, it can really slow things down.

How Websites Are Displayed Using The Pi-hole

Depending on the Website and the content you decided on for your index.html  file, you may see different results.  Sometimes, the ad-space is completely shrunk down as shown below.

pi-hole-example1

But other times, a box or blank whitespace may still remain where the ads used to be.

pi-hole-example2

Try changing the index.html  file to use different methods and see how things work for you.  It’s likely that the Pi-hole won’t be able to block every ad perfectly, but it does work for the most part.  There Internet is organic and advertisers will always find a way to inject ads.

Success!

You can test it at AdsBlocker Testing or Simple Ad Block’s test.

If you want a real-life test, I would suggest Cafemom or Adsreview (from the videos at the beginning of this post). Any article from Macworld or OSX Daily are also some of the sites I use to test as they have a lot of ads.

Some Ads Are Still Showing

Ads can be served from many different domains and the lists may not know about all of them.  The first thing to do is find out what domains are being used on a certain site, and then add/remove domains from your list.  The easiest way to do this is using DNSThingy’s Whitelist Assistant for Chrome.  Details on how to use it for this purpose can be found here.

Watching the Log File And Viewing Perfomance

When we edited the dnsmasq.conf  file, we added the debug option (log-queries ).  This is very useful for finding out what URLs ads might be coming from.  You can simply watch the log file as you navigate to a site on your computer.  To watch the log file in real time, use this command.  It is kind of fun to see what pops up when you navigate to a site.

tail -f /var/log/daemon.log

But since we added the line log-facility=/var/log/pihole.log , the path just needs to be adjusted.  If you installed the Pi-hole before August 2015, the command above will still work.

tail -f /var/log/pihole.log

Once you find a URL you want to block, you can just append it to your /etc/pihole/blacklist.txt.

An LCD Screen For Viewing In Real-time What Ads Are Being Blocked

Using this log file, you can make a real-time monitor for the Pi-hole that lets you see what domains are being blocked.  I am using a 2.8″ LCD screen so I can just peek at it while browsing.  This real-time monitor is also available over ssh .  See the video below of it in action.

Web Interface For Viewing Your Pi-hole’s Performance

The automated installer includes a nice Web interface, which shows stats from a Web page.  You can manually install it.  Here is a screenshot:

pihole whitelist

Donations For Continued Development

The Pi-hole is free, but powered by your donations.  I mostly work on it for fun in my free time, but when I do receive donations, I push myself to do some extra development.




Donate Bitcoins

 

Author: Jacob Salmela

I am introverted.

  • Dave RC

    Hi. First of all, thanks for the great work. I have been using the Pi-Hole with adList.conf for a while now, but your new way of doing it with the HOSTS file seems so much smarter.
    Unfortunately, when I run the new gravity.sh script, it stops and reads:
    “/usr/local/bin/gravity.sh: fork: Cannot allocate Memory”
    I’m using a Raspberry PI Model B, first edition.
    Thanks in advance.
    David

    • jacobsalmela

      Two things to try, both via sudo raspi-config:
      1. Expand the filesystem so there is enough storage space.
      2. Allocate less memory to the GPU since it’s not needed for the Pi-hole.
      I am going to test it out on my B+ and see if I can replicate it.

      • Dave RC

        Thanks for the hints. I had already donw both. But after I stopped lighttpd and dnsmasq it worked and put together the hosts file.
        But after using it for a few minutes (not even hours), I felt the performance was very bad. It still resolved all addresses fine, and blocked ads as it was supposed to, but the time until a page was actually displayed was very long. Way slower than with adList.conf, like I used it before.
        Can you confirm that Performance on RPi B is slower now? It was a fresh Raspbian install.

        • jacobsalmela

          My experience has been quite different. I have been using it for a few days without any problems.

          The first thing that comes to mind is that you are possibly using both /etc/dnsmasq.d/adList.conf and /etc/hosts, which would definitely cause performance issues. The solution would be to remove adList.conf.

          A few other things: How fast is your SD card? Also, what is the load average from the uptime command?

          • Dave RC

            Hi. For now I have suspended the PI-Hole project. It worked fairly well the past few days, but still slower than it was when I used the adList.conf.
            I can definitely confirm that adList.conf has not been used. It probably is the slow SD card. I also tried running the PI-Hole without lighttpd and moderate overclocking.
            I will try again in the next few weeks. Keep up the good work, and thanks for your effort.

          • jacobsalmela

            It’s partially due to the amount of domains on the lists controlled by the other sites. ~120,000 seemed to be the sweet spot. Once it got higher than that, the hosts format performed better. But a faster SD card can make a difference…

          • Gryphraff Gryphon

            I had the same issue with “Cannot Allocate Memory” – stopping lighttpd and dnsmasq during update fixed this for me.

            I’m running on an original Model B, 16MB GPU space and a 8GB expanded SD card. I may update to a B2 later, but it’s older cousin seems to be doing fine at the moment.

          • jacobsalmela

            So did you stop dnsmasq and lighttpd prior to running the gravity.sh? Just curious if this is something I should incorporate into the script so users do not run into this.

            I also use the Pi-hole on the original B with 16MB GPU space and expanded SD card. It has been trucking along for many months now. Although, executing commands and connecting over ssh shows a noticeable lag. It’s not a big deal though since I dedicated my B to be a Pi-hole and nothing else. I am using the Pi2 for continuing development.

          • Gryphraff Gryphon

            Yes, I stopped both before running gravity.sh, using /etc/init.d/ stop and then start when gravity.sh was done.

    • jacobsalmela

      I just tried it on my B (it’s actually not a + as I mentioned) and it worked fine. Did you run this on a clean install of Raspbian? Or do you have other services running?

    • xayide

      The problem you are encountering is when gravity downloads/parses the large mahakala block list. I had this on my Windows server since the VM only had 1 Gbyte of RAM and it is used for a lot more than pihole. So I solved it temporarily by disabling my kodi-server. long term fix is to turn of the VM and put some more memory to it. So to confirm just edit gravity.sh and remove the mahakal list and see if that works. If it does, I would suggest see if you can gather some more free ram somehow.

  • Pat

    Shouldn’t the URL to download gravity.sh be “https://raw.githubusercontent.com/jacobsalmela/pi-hole/master/gravity.sh” instead of “https://github.com/jacobsalmela/pi-hole/blob/master/gravity.sh” so you don’t get all of the HTML formatting from github?

    • jacobsalmela

      Yes. It should. I believe everywhere I linked points to the raw format…where did you see that URL? If you know where, I will fix it.

      The Github page shows the correct one.

      • Pat

        Its on this page under the “Create A Script To Download The Lists And Save It As The Hosts File” section

        • jacobsalmela

          Thanks. I fixed it. That’s what I get for self-editing a 6,000+ word article.

  • Halp

    Hello. After installing Pi-Hole, whenever I do things like apt-get update/upgrade or anything ‘sudo’ related, I get the message ”sudo: unable to resolve host”. Do you know how I can resolve this?

    • jacobsalmela

      I have had this happen, too. This line in the script fixes it, but if your Pi is named something else this line will need to be different. This will be resolved in a future push, but in the meantime, you can change that line to this:

      echo "127.0.0.1 $(hostname)" | cat - $origin/$accretionDisc > $origin/latent.$accretionDisc && mv $origin/latent.$accretionDisc $origin/$accretionDisc

      I think the issue has something to do with one of the settings in dnsmasq.conf, but I don’t quite know what causes it yet. I used the solution above and it eliminates that message.

  • Halp

    Is it possible to get this to work over a private VPN setup? At the moment I’m using the Zeitgeist tutorial (https://www.zeitgeist.se/2013/11/22/strongswan-howto-create-your-own-vpn/ ) and want the benefit of ad-blocking while I’m out and about with a VPN enabled.

    • jacobsalmela

      There is no reason it shouldn’t. Once your on the network, you can use the Pi as your DNS…I haven’t tried it though.

      • Halp

        Actually, I was able to add it as a DNS server in my /etc/ipsec.conf file. It’s been working so far except for times when I’m connected to the VPN, but ads still find their way through.

  • James

    How much of the Pi’s grunt does this need? I’d like to do other things on my Pi. Can this be done with Apache instead of Lighttpd?

    • jacobsalmela

      You can use Apache or any other Webserver. I chose lighttpd for performance reasons. The goal was to use the smallest impact so it always runs as fast as possible.

      I think the original B Pi will struggle doing other things as I notice a slowdown once I open the Pi-hole, but I have been using it on my Pi 2 and I have been able to have it doing other things without issue. You can also get more/less performance based on your SD card, overclocking, and other settings like that.

  • Just dropping by to say Thanks and give a big thumbs up. Installed it yesterday and we all have seen an improvement in our home. Thanks for all the good work!

    • jacobsalmela

      That’s great! I love to hear that it’s working for you. Thanks for the note.

  • JE

    The second version is even better than the first one. Thanks for a great job ! It gives a strange feeling of satisfaction to be able to block out those add’s. It goes beyond the mere technical improvement.
    Do you have an Amazon wish list or a preferred charity where I can express my gratitude in a small but more tangible format ?

    One very small suggestion : why not adding a reboot to the manual, just before the launch of the script ? I changed IP and host name before launching the automatic installer. The host file now has two IP addresses for each entry : the static one and the dynamic one.
    Eg. The first line reads : 192.168.1.10 192.168.1.30 0000007.ru where .10 is the static one, and the .30 is the dhcp address the static one should have replaced.

    Is it possible that some websites stop functioning ? I have one example : Kat.cr.

    • jacobsalmela

      Thanks so much for the donation. It’s very motivating and satisfying to know you like it!

      I had that same issue of two IPs in the list. You should be able to just run gravity.sh again once you have only a static IP to resolve that issue.

      As for the 127.0.0.1 raspberrypi:

      You can either download the latest script from Github, or just change this line so it reads $(hostname) instead of raspberrypi:

      echo "127.0.0.1 $(hostname)" | cat - $origin/$accretionDisc > $origin/latent.$accretionDisc && mv $origin/latent.$accretionDisc $origin/$accretionDisc

      This will grab whatever your hostname is and make an entry for that instead of raspberrypi.

      Yes, some legitimate sites have been blocked. To solve this, just add a /etc/pihole/whitelist.txt with one domain per line and then run gravity.sh again. It will detect the whitelist and apply them.

      I’ll definitely be doing more work on the Pi-hole. I tried to include as many links to other resources as possible, while still making it easy for most people to follow and setup, but it’s difficult because everyone is at different knowledge levels. I will definitely look at adding some basic configs for dnsmasq.

      Thanks again for the donation!!

      • JE

        Jacob : it works 🙂
        One last question : can I still use the dnsmasq as my internal dns ? Where do I have to create the hosts without them being overwritten when gravity is run ?

        Thanks,

        Joris

        • jacobsalmela

          As of now, gravity.sh will always overwrite the existing hosts file. You could modify your script to add the additional hosts in by replicating one of these lines in the script, but put your own hosts and addresses there. It’s not perfect, but until I can do some more development, that’s probably the best bet.

        • Here is how i did this:
          in /etc/dnsmaq.conf i added these lines:

          domain=local
          addn-hosts=/etc/hosts.local

          in /etc/hosts.local i keyed in the ip adresses en hostnames of the static devices on my network, (routers, switches, mediaplayes and servers).

          This keeps working even after a run of gravity.sh. It just uses the dnsmasq part.

          • jacobsalmela

            That’s a good way to do it. It segregates your local hosts from the ones gravity.sh gets, but still allows both to coexist without worrying about getting overwritten.

            I’ll have to edit the post to mention this.

          • You might want to replace de .local i used with .lan in the example. Usage of .local seems to have some strange conseqences. Reconfiguring to .lan as we speak and i will let you know.

          • JE

            Did this (with a .lan extension). Works great. Thanks !

          • jacobsalmela

            Sweet. Yeah, I think .local is used with Bonjour and mDNS so it’s probably something related to that. I recall this because I set the Pi to be raspberrypi.local using avahi-daemon.

            Guess I have more work to do to keep up with the improvements!

          • rez

            To handle my local address I added these lines to /etc/dnsmasq.conf:

            server=//
            server=/.local/

            These server lines are in addition to the two that are already there for dns lookup. The first server line is for unqualified addresses (the ones without .’s in them) and the second is for any .local addresses that are entered and direct the pi to ask query the router for the dns info. This allows seemless access as it was before the pi hole for any dynamic clients

            I set my router to hand out the pi’s ip address for dns and to not hand out itself for dns. The result is the pi hole receives all dns requests and then if it a local address it asks the router and if it is not a local address, it either blocks it or passes it up to the “real” dns servers.

      • With regards to the double ip adresses, i found that my raspbian keeps asking for a dhcp ipaddress although i have configured it to use a static address. hostname -I keeps giving two. I fixed it quick and dirty, and replaced the “Hostname -I” in the gravity.sh script with the static ip-adress and a space ( “10.10.10.10 ” )

        • jacobsalmela

          I wonder if the network interface needs to be brought down and up again, or possibly even a restart. The people I have talked to said it is always the DHCP address and the new static one when there are double entries in the hosts files. So a restart would probably fix that before trying to run gravity.sh

          • Godisard

            Static IP should/can be set different with the latest Raspbian Wheezy image (2015-05-05). You have to edit and put static ip in /etc/dhcpcd.conf and leave /etc/network/interfaces at default. Alternatively you can edit and use /etc/network/interfaces for static ip but then you should disable the dhcpcd daemon: sudo update-rc.d -f dhcpcd remove.
            Otherwise you will indeed have a static and dynamic ip.

          • sudo update-rc.d -f dhcpcd remove worked like a charm!

          • jacobsalmela

            So if you disable the dhcpd daemon, it won’t even try to get a DHCP address? That might be the way to go so there is never a worry about it…

            Interesting. Thanks!

          • Well it has had a few reboots in the process, but during boot i can still see it asking for an ip on boot. And hostnames -I keeps giving both. Hostname -i just gives the static one. But since i didn’t want to spend to much time deinstalling dhcp client and whatnot i opted for keying the address in in the gravity.sh

          • jacobsalmela

            I have noticed that if you are using IPv6, hostname -I displays those, too. I’m looking to add IPv6 support. I had it partially working when I wrote this article, but it just wasn’t working the way I wanted, so I put it off.

          • JE

            Tested this out during a reinstall with Raspian on a Pi2. After configuring the Linux part with hostname and static IP, the machine was rebooted. Then I started the PI-hole install. It did not make a difference : the old name still popped up in the host file.

          • jacobsalmela

            So if you run just the hostname command, does it show only the correct hostname?

            Also, did you download the new script?

            How are you setting the hostname?

  • robbo

    I setup and it ran perfectly for a few hours.
    Then the raspberry pi2 powered down/
    When i went to see why (it was still plugged in to electricity) i disconnected and reconnected to reboot. But nothing happened. when i pressed SD card to eject it burnt my finger. Actually burnt my finger. Upon looking at SD card there was little deformation on card due to heat.

    • jacobsalmela

      It sounds like some sort of hardware failure or a bad SD card. What SD card did you use?

  • Brad Hopper

    Hey, the previous version was working great for me if a little slow on my RPiB+ I’m now running the latest on RPi2 and it’s even faster, but seems to not redirect ad links which are in fact in the hosts file. Some are redirected others are not. For example, on finance.google.com, there is one small ad over on the right, I’ve been using as a test case. This has a link pointing to ad-g.doubleclick.net which throws an error. The javascript console shows “connection refused”. I also see other doubleclick and google ad urls which are in the hosts file which are just passed right through without any errors.

    • jacobsalmela

      When testing the Pi-hole, I typically saw the “connection refused” message when lighttpd was not setup properly. What happens if you navigate to ad-g.doubleclick.net in a Web browser? If you get your blank page, then you know it’s working. But if you get another connection refused, you should look at the /etc/lighttpd.conf file and also make sure the domain is in /etc/hosts. I’ll test it out on my own Pi-hole to see if I can replicate the issue.

      • Brad Hopper

        Navigating to that url does result in a blank page, and many /most? of the ad domains are blocked, but what I find is that some of them are simply not blocked. Now, I haven’t spent as much time using the pihole2 directly as the preferred dns in the client network setting, since that worked well with pihole1, instead working with putting the pihole into the preferred dns of the router itself, so all tools connected to it will get the benefit. Now that just didn’t work at all with my dlink router, but I replaced it with a newer tplink one which does seem to work, but intermittently as I described, with some ads blocked, some ignored, and others causing errors. BTW, I used your “all in one” installation so I hope I didn’t somehow muck up the lighttpd installation process. But there is another factor in there, and I’ve been wondering if somehow the router times out if the rpi doesn’t respond fast enough and then the ad just goes through. Confusingly, the tail command actually reports the non-redirect of the ad to its intended destination. Are there any other diagnostics to see what might be happening under the covers?

        • Godisard

          Is it perhaps possible that your client pc is using the secondary dns of your router instead of the primary dns (the pihole) in cases where you see ads coming through? At least that’s what I experienced on my Macs. Like you I had my pihole set as the primary dns server of the router but ads still got through randomly on clients. After some searching I found out that in OSX primary and secondary dns servers are apparently treated equal (since Snow Leopard), i.e. on a Mac dns server ordering is not honoured but search order is in fact dynamic. So my clients were randomly using the secondary dns of the router which was an OpenDNS server.
          I don’t know how Windows and Linux treat dns server ordering though.
          Anyway, my solution was to set up a second pihole. Since about a week I’m running two piholes as (local) dns servers for my router and I haven’t seen a single ad since on any of my clients.
          I have to say that what Jacob has created with the pihole is one of the best uses for a Raspberry Pi I have come across yet. Very easy to set up, very reliable running (haven’t rebooted the pi’s since setup) resulting in incredibly fast and ad free Internet use for all my clients including an iphone and ipad.
          I’m enjoying it so much that I just made a small donation to express my thanks and hope he keeps developing it to make it even better.

          • jacobsalmela

            Interesting. I’ll have to look into how OS X dynamically uses DNS servers. I do know that if you are using a wired and wireless connection the traffic will use either interface (usually on whatever one the request was made).

            I was working on the real-time LCD display for a while and have it working, but I figure the Web interface would be more useful to people. So I am working on that now. It would be nice to navigate to your Pi via a Web browser and see stats, restart DNS, etc.

            Thanks again!

        • jacobsalmela

          Since you get the blank page when navigating to the domain in a browser, it means the hosts file is working properly.

          Some other tools I use for testing and diagnosis are:

          curl -I someaddomain.com

          This will return the header of the domain, which will show if your redirection is working or not. I also add an X-Pi-hole header in the lighttpd config file so I know for certain that it is working.

          dig somaddomain.com

          Just check to verify that your Pi is responding in the ANSWER section and not some other server.

          You may also want to use traceroute or nslookup.

          Since some ads are not blocked, you may want to experiment with using a different Webpage to take the place of ads as each browser/page may render it differently.

          There is also the issue that ads are only being blocked by a simple Webpage or image file, so if the page is expecting video, audio, Flash, or some other content, that may also cause issues. So my best suggestion is to try the different options for the blank page and see how the page displays.
          It will also be a good idea to flush your browsers cache and any DNS cache to resolve any random issues.

          Finally, as for the router, are you actually installing it on the router? If you simply change the DHCP settings to give your clients the Pi’s IP address as their DNS server, you do not need to change it manually for each device. When they get an IP address, the DHCP options will come with it.

  • watyo

    help – i am new with linux but managed to follow all the manual steps (want to learn the basic) using default settings. so far, when i am on my pi and pinging 247media.com and it returned my pi ip. but not on my window laptop.

    do i need to change the dns ip on my pi?

    little background:

    1 – i am using wire for my pi and wireless for my laptop (laptop ipv4 dns is configured and disable ipv6)

    2 – when i checked the services on my pi, i noticed the (-) dnsmaq so i am assuming it not running.

    3. i am using the 25bytes.gif to replace the ads.

    any ideas???

    Thanks

    • jacobsalmela

      If you can ping 247media.com and get your Pi’s IP address, then both dnsmasq and lighttpd are working. So yes, you need to change the DNS server of your Windows laptop to point to the Pi. You also need to do this for every device on your network that you want to use the Pi-hole with unless you make the change on your wireless router.

      Check if dnsmasq is running with this command:

      sudo service dnsmasq status

      One of the best real-life tests to see if it is working or not is this crazy site.

  • Mike Wood

    Wow! Installing this from start to finish took less than an hour and what a huge improvement! I’ve used the Pi-Hole as the DNS on my router so all clients see the benefit.

    One question. I used the ‘quick install’ scripts and I’m not sure if that sets up crontab to periodically run gravity.sh. I don’t think it does (at least not as root) but I wanted to check it was still necessary before adding it manually?

    (If it is necessary, can I suggest you add it to the automated install.)

    Thanks again – made a donation already I was so impressed 🙂

    • jacobsalmela

      I appreciate that very much. The donations are what keep the project going. Plus, it’s always satisfying to hear that people enjoy it.

      As of now, the crontab is not deployed with the automatic script, so you will have to manually add it with

      sudo crontab -e

      I’m considering one of these three options for deploying it with the install script, but I haven’t decided which one will work best or at what interval to check.

      • Mike Wood

        Thanks, I’ve added the entry to crontab with a 1 week schedule.

        Everyone in the family has commented on the improved speed of browsing. It’s amazing what a tiny Raspberry Pi is capable of. I’m currently using an old Model B that I had lying around from a prior project, but plan to replace it this weekend with two Model 2 devices providing redundant primary and secondary DNS to the router.

        I’ve three or four Pi systems in use in the house already and have had Micro SD cards fail in the past. I now know enough about the limitations of the SD card filing system, particularly with a heavy read/write app like a caching DNS, to not trust a single device to anything critical! However the cost of two Pi 2 is so minimal that it still makes enormous sense.

        I’d been looking at adding something like this to my router code – but this is a much better solution. Thanks again!

        • jacobsalmela

          Yeah, I have been using mine for about three months and definitely notice some speed improvements (in addition to blocking the ads).

          I try to limit the writing to the SD card, by only downloading the lists if there have been changes.

          If you want, you can also store the lists in RAM instead of writing them to the card by changing the origin to be /run/shm instead of /etc/pihole.

          Since there were some issues when one of the lists updated, I wanted to have the lists in place so there was something to revert to. In addition, it makes it easier for me to debug any issues as you can go back and look at what list a certain domain was on.

          It’s still a work in progress, but I have big plans for it!

          Thanks again. I’m really glad you enjoy it.

          • Mike Wood

            Update. I’ve now set up two Pi 2s running Pi-Hole configured as primary and secondary redundant DNS to the router. (EdgeRouter Lite). Working excellently.

          • jacobsalmela

            Solid! That sounds awesome.

          • andi

            May I ask you, how you configured it, so that both ad-blocking and DNS-Caching works? I have 1 DSL Router(FritzBox), it’s DNS Server is set to the PiHole in the internal network.
            It blocks ads now. (Yeah! 😀 )This is my first great success, i am really glad it works. No need to manually set the DNS on each device, this is great. 😀

            But something about the DNS Caching does not work, as it always requests them online on the 2 OpenDNS Servers (I checked the log/tried ” $ dig github.com”) instead of replying cached results.
            It’s basically:
            Laptop > DSLRouter > PiHole > OpenDNS

            Funnily enough, when i “$ dig @pihole github.com”, the second reply gets the cached, fast answer from the PiHoles DNS Cache. (But the PiHole forgets it again after a few minutes…)

            Gratulations to your working setup, by the way! I can appreciate it, as I am totally lost with 1 DNS-Server already, if I tried to setup 2 at the same time, I think I would ruin the network completely. 😀

          • jacobsalmela

            The cache should stay until dnsmasq is stopped/restarted or you restart the Pi. The first query will always be slower because the Pi does not know about it yet.

          • Mike Wood

            That sounds like you may have the cache size set too small in the dnsmasq.conf file. The default is only 150 so it will quickly overwrite old entries.

            Make sure that you have ‘cache-size=10000’ in /etc/dnsmasq.conf

            The only reason I have two of the Pis set up is to provide redundancy, I have cron tasks to reboot and update the gravity files at different times. It’s not really necessary.

          • Mike Wood

            This sounds like you may have the cache size set too small (or not set at all so it defaults to 150) in dnsmasq-conf and it is quickly getting overwritten with newer entries. Make sure you have the line cache-size=10000.

  • xayide

    I do not get all those 900.000 domains. I only get like 430.000 unique….Hs mahakal list been cut in half or do I miss anything?

    ** Aggregating list of domains…
    ** Blacklisting 3 domain(s)…
    ** Whitelisting 28 domain(s)…
    ** 631331 domains being pulled in by gravity…
    ** 432352 unique domains trapped in the event horizon.
    ** Formatting domains into a HOSTS file..

  • watyo

    hi – anyone has performance issues when a request timed out or failed to respond. for example:

    http://thestir.cafemom.com/entertainment/158359/glee_changes_already_under_way?utm_medium=sem2&utm_campaign=prism&utm_source=outbrain&utm_content=0

    the test page took longer than 2 minutes to render to due a bunch of requests failed to respond.

    • xayide

      Loaded in 2 secs here.

    • jacobsalmela

      Did you set it up manually or use the automated installer?

      It sounds like the DNS queries are not being answered, so either dnsmasq isn’t working or lighttpd isn’t redirecting them.

      • watyo

        i found what i did wrong. when i set the static ip I entered ‘1 ‘instead of ‘zero’. how can i change the piholeIP=$(hostname -I) to the new ip?

        • watyo

          never mind. i ran the script again

          • jacobsalmela

            Yup. Once you change your IP address, just run gravity.sh again and it will rebuild the hosts file from scratch.

  • Halp

    Hey J. Awesome stuff you’re doing with the Pi-Hole project. I managed to install it on my RPi 2, running a VPN setup and it works great with it. I have a question for you though. Is there any possibility that we’ll see video adblocking implemented into Pi-Hole to block stuff like YouTube ads for example?

    • jacobsalmela

      People have been asking if it works via VPN. I have been telling them there is no reason it shouldn’t, so I’m glad you can confirm. I just haven’t made time to write up a post on it,

      As far as the video blocking goes, it’s a long work in progress. At one point, I was able to skip the Hulu ads completely, but haven’t been able to replicate the success.

      Also, one person claims to have been able to block them on Xbox 360/One, but my results have been inconsistent.

      To sum it up, I’m on the right track and I have learned a lot about how Hulu serves the video ads, but it’s a larger commitment than blocking simple image ads and since the Pi-hole is powered by donations , it’s not something I can do full time yet. =( Maybe someday…

  • Matthew Johnson

    I really appreciate all the work that you have put into your pihole project. I’ve been a user of version 1 and now of version 2. I have the following bug report:
    – I have a raspberry pi b+ running pihole version 2
    – pihole version 2 was installed using noobs and the automated install link you provided in the version 2 walkthrough
    – when the script executes, the following error is raised:
    pi@raspberrypi ~ $ sudo /usr/local/bin/gravity.sh
    ** Neutrino emissions detected…
    Skipping adaway.org list because it does not have any new entries…
    Skipping adblock.gjtech.net list because it does not have any new entries…
    Getting adblock.mahakala.is list…
    /usr/local/bin/gravity.sh: fork: Cannot allocate memory
    pi@raspberrypi ~ $

    Is this ‘Cannot allocate memory’ failure due to the size of the ‘adblock.mahakala.is’ list? If so, how would I be able to work around the memory failure without commenting out the list?

    Thank you for all your development work with this very useful tool that allows me to block all ads within my home on both desktop pcs and android devices without having to root them.

    Regards

    • jacobsalmela

      Yes, the mahakala list is very large and can cause this error message. 90% of the time, the fix is to allocate less memory to the GUI via the raspi-config menu.

      Assuming you aren’t using the Pi for anything else, set it as low as possible. If you are using the GUI, you might have to play with the values to find the right balance. You could also modify the script to always write the mahakala list to disk, but I’m not sure if that is worth it or not.

      I appreciate your kind words. It’s satisfying to know people enjoy it and motivates me to keep working on it.

      Let me know if that fixed your error message.

      • Matthew Johnson

        I am not using the pi for anything else other than this pihole application. I have set the ‘gpu_mem’ to ’16’ in /boot/config.txt. I still receive the ‘Cannot allocate memory’ error. I wanted to forward along these results such that it might help make the script more robust for others / help with development etc. I understand that I could work to modify the script to write data differently, or I could split it up and write one script to make the list, and another to create the hosts file. Wanted to forward along however, that I’ve some confidence that for a raspberry pi b+, when utilizing the automated install scripts (fresh install) a memory allocation error will be thrown during list gathering portion of the gravity script. I’ll rock along for the moment with ‘mahakala’ commented out for a bit as I’m preparing for college fall classes this semester. Thank you for your time Jacob!

        • ewg1967

          I am also getting the “/usr/local/bin/gravity.sh: fork: Cannot allocate memory” message. I am only using the pi for this application only and have the gnu_mem set to 16. Any suggestions?

          • jacobsalmela

            It seems the mahakala list is getting bigger every day. I think I will need to look at possibly just writing it to the disk every time to avoid this error.

            I have opened an issue for this.

        • Harald Kroiss

          I found out the at my DietPi version the Swap-Size is limited to 100MB, i set it to 500MB and afterwards was working fine 😉

          • jacobsalmela

            That was one of the other fixes for Raspbian, too. I will have to integrate this into the logic of the script and/or the installable image. Thanks!

  • Harald Kroiss

    Hey, first of all: NICE PROJECT , keep it up man 😉

    installed it yesterday on my Raspi B+ and works good, i only have one problem right now:

    When i open youtube, every second video isn’t working, i can see the normal HTML5 Player interface but the video is not working. The Video field just stays black for like 5 seconds and then the video is playing, some are playing from the start fine.

    Wait a second, right now i can imagine why… its the time where the ad is playing normally? isn’t it?

    can this be true? is there a way to completley block the ads to skip the black timeout?

    no kidding just realized this writing this comment 😉

    • jacobsalmela

      Blocking videos is still a work in progress, but yes, it’s just blocking it out for the time that the ad would normally run. Eventually, the goal is to skip them completely, but it will involve some more trickery.

  • digitlman

    The whitelist function does not work for me. I put the “www.domain.com” I want to exclude into /etc/pihole/whitelist.txt, run gravity, and it still blocks the domain.

    • jacobsalmela

      You probably need to add http://www.domain.com and domain.com (without the www). The script doesn’t know how to parse those out just yet. You can also run this command to see what domains you need to whitelist:

      cat /etc/pihole/gravity.list | grep domain

      • digitlman

        Ok, did that. Still not working. Does not show up in the gravity.list.

        • jacobsalmela

          What is the domain name? I will see what list it is on and if I can get to it.

          • digitlman

            One of them is pornhub.com. Yeah, I know ;-).

          • jacobsalmela

            Entries exist in these lists: winhelp2002, hosts-file, and mahakala. You may grep the domain from each of those files using the command mentioned previously and then put them in your /etc/pihole/whitelist.txt. Some domains might use a separate subdomain, which may also need to be whitelisted.

            In Chrome or Firefox, you can also right-click a video or other element and choose Inspect Element, which may reveal what domain needs to be whitelisted.

            If none of those things work, then you may have to see if the site’s content is being delivered from some other domain not even related to the site.

  • Mikkel

    Hi! similar to digitlman, I am trying to add a domain to whitelist.txt, which is not working. I do see that running gravity.sh creates a ‘latentWhitelist.txt’ file with the domain I am whitelisting. However, running gravity.sh generates this output (among others): ** Whitelisting 0 domain(s)…

    Any idea why it’s not working? The domain I want to whitelist is: ad.zanox.com

    • Mikkel

      Found out something strange, I *think* whitelist.txt won’t notice the first line in the txt file: when I add more domains (more lines) it works…!

      • jacobsalmela

        In my testing, this commit fixes the issue.

        • Tabatha Altland

          Nope … replaced the gravity.sh with the newest version as you suggested and it doesn’t work. I’ve added ebates.com and http://www.ebates.com, skipped the first line as well . When ran gravity.sh script does show the domains being whitelisted, but when tried from any PC it shows blacklisted by pihole message.

          ** Aggregating list of domains…
          ** Whitelisting 4 domains…
          ** Whitelisting 8 ad list sources…
          ** 1634094 domains being pulled in by gravity…

          root@dietpi:~# cat /etc/pihole/gravity.list | grep ebates.com
          192.168.2.125 adultrebates.com
          192.168.2.125 apprebates.com
          192.168.2.125 certificaterebates.com
          192.168.2.125 com.shc-rebates.com
          192.168.2.125 erebates.com
          192.168.2.125 joebates.com
          192.168.2.125 pornrebates.com
          192.168.2.125 shc-rebates.com
          192.168.2.125 toprebates.com
          192.168.2.125 http://www.certificaterebates.co
          192.168.2.125 http://www.shc-reba
          192.168.2.125 http://www.toprebates.com

          • jacobsalmela

            Your script output shows that ebates.com is whitelisted as there is no entry for it. Did you mean the blacklist instead?

    • jacobsalmela

      Yeah it definitely does not seem to be working properly. With just ad.zaonx.com in the whitelist, I get this:

      cat /etc/pihole/gravity.list | grep zanox

      192.168.1.121 api.zanox.com
      192.168.1.121 api.zanox.ws
      192.168.1.121 static.zanox.com
      192.168.1.121 ui.zanox-affiliate.de
      192.168.1.121 ui.zanox.com
      192.168.1.121 www.zanox-affiliate.de
      192.168.1.121 zanox-affiliate.de
      192.168.1.121 zanox.com

      and with ad.zanox.com and zanox.com in the whitelist:

      cat /etc/pihole/gravity.list | grep zanox

      192.168.1.121 api.zanox.ws
      192.168.1.121 ui.zanox-affiliate.de
      192.168.1.121 www.zanox-affiliate.de
      192.168.1.121 zanox-affiliate.de

      I’m sure it has something to do with the regex used to remove them…Maybe something like this will fix it?

  • Glasofruix

    Hey there, i’ve installed everything as described but it doesn’t seem to work, like at all. The admin console shows no dns queries answered or ads blocked (i tried every browser i have, emptied the dns cache etc…).

    • jacobsalmela

      Did you install manually or use the DietPi image? If you used the image, then you just need to run dietpi-update to fix it. If you did it manually, check to see which log you used: the default, /var/log/daemon.log or the new one <code/var/log/pihole.log.

      • Glasofruix

        Using the dietpi image, i just reinstalled everything from scratch just in case but the problem persists, dietpi-update does nothing it just says that it’s up to date.

        • jacobsalmela

          Did you remember to point your devices to use the Pi as their primary DNS server?

          • Glasofruix

            Yes, also emptied windows dns cache just in case and rebooted the computer.

          • jacobsalmela

            You might need to use some more advanced tools like traceroute, dig, or nslookup to determine where the failure is. You could also watch the log file with

            tail -f /var/log/pihole.log

            Other questions to help narrow down the issue: Is name resolution working? Is the Pi reachable (i.e. ping)?

            However, it seems like it should be something simple…maybe it’s simply a bad cable from your router to the Pi? Or maybe your Pi changed IP addresses if you didn’t set a static one. There are lots of variables to try, but start with the simplest.

          • Glasofruix

            The pi has a static ip address, the first thing i did actually. It’s reacheable via ping and i’ve done the installation via ssh, so it’s really not the cable (since i also have internet access when using it as dns and not when it’s powered off). Traceroute shows nothing of interest since the first hop is my .1 router. I’ll try with nslookup and dig tommorrow, i’m a bit fed up with the thing at the moment :p

  • Karl Hudgell

    Hey, I have everything working now, but don’t understand how to get the RTM showing on my PITFT?

    Any help would be great 🙂 thanks

    • jacobsalmela

      You will need to use chronometer.sh, which is a script that I don’t really have a writeup for yet as it was just a quick side project that turned into the Web interface instead.

      You will need to install figlet, toilet, and bc in order for the script to run. Then, you will probably want to have the pi user log in automatically. After that, you need to run the script via /etc/profile or the pi user’s .bashrc.

      • Karl Hudgell

        Wicked, I will have a play with it, is the webui about?

        • jacobsalmela

          /usr/local/bin/chronometer.sh is the location of the script. The Web interface is just a pretty way to view some of the stats and also has some donate links if you are feeling generous. Have fun!

          • Karl Hudgell

            I seem to have got it all working with the screen on boot, but i wanted to ask, it seems like the chronometer app is restarting ever so seconds, is that by intention to refreshing? or am I doing something wrong?

            thanks

          • jacobsalmela

            Yes, it’s just in a infinite loop that reads the stats every few seconds. There might be a better way to do it, but that’s what I made so far. I kind of abandoned the LCD in favor of the Web GUI. Although, it is still pretty fun to have it show up on the LCD. Long term, I thought about using it for a standalone hardware device…

          • Karl Hudgell

            ok cool, but the web ui is not functional yet from what I can see?

          • jacobsalmela

            The Web interface works–I’m not certain where people are running into issues. It only shows entries from the current day.

          • Karl Hudgell

            Ok, I didn’t even try it, as I wasn’t too sure what port it is using? as I can’t see it listed anywhere and port 80 is used by the blank page?

    • Martin

      RTF? PiTFT ?

      • Karl Hudgell

        RTM – Real Time Monitor
        PiTFT – Its a screen running from the GPIO pins on the pi, like shown in his video

  • Martin

    Hi

    I set this up on a straight debian install.
    Had to do a few extra things:

    1. install sudo, curl
    2. add ‘pi’ user using ‘adduser pi’
    3. change gravity.sh (kill -HUP $(pidof dnsmasq) to kill -1 $(pidof dnsmasq)

    • jacobsalmela

      Is it running on a regular computer (i.e. not a Raspberry Pi)?

      The script can certainly be modified to run on most UNIX-like systems with some knowhow. I am focused on the Pi since you can plug it into your router and forget about it while it just does it’s job of blocking ads.

  • dv8ed

    Hi Jacob! I was using Privoxy but hands down this is way better, faster, stable and blocks more ads. I was testing this on my Raspberry Pi Model B rev 2 (512MB). Ordering a Raspberry 2 Quad Core 1GB. Does Pi Hole write to microSD card a lot? I was thinking of ordering mSATA to help with write and improve speed or is it not necessary? I don’t mind having to replace microSD card every year if it will last that long they are cheap.

    • jacobsalmela

      I have tried to minimize writing to the SD card by only downloading the lists if there have been changes, but I have also had to increase the swap. I have been using my Pi-hole for many months without a failure, but yes, ideally, I would like to have everything off of the SD card and only use it for booting.

  • Chris Hawkins

    Any plans/possibilities of getting this working with the Adafruit 16×2 LCD screen?

    https://learn.adafruit.com/adafruit-16×2-character-lcd-plus-keypad-for-raspberry-pi/overview

    Has buttons and stuff to navigate menus 🙂

    • jacobsalmela

      Yes! I have one purchased but haven’t assembled it yet. It’s lower on the priority list, but there. I would love to do more in a shorter amount of time, but since it’s only funded via donations, and my desire to contribute to the open source community, things progress slowly.

      • Chris Hawkins

        Fantastic, will see if I can maybe assist with a donation 🙂 Love the project though top work.
        Would be good to see more tools and things in the admin interface. Maybe some statistics of possible like average DNS response time (if even possible)

        • jacobsalmela

          It’s all in the works. I usually do most of it on the weekend in my free time. New features and bug fixes are a bit slow, but steady.

  • jacobsalmela

    Yes, I am definitely planning on making a Pi-hole Access Point, and a Pi-hole VPN. Many projects, no money to make a living off of it.

    Having the bridge will definitely make things more confusing. I can’t offer much advice at the moment since I haven’t even dug into trying this yet. Let me know if you figure something out though.

  • Zachary Ferkul

    Hey set it up but when i go to the config page on my browser this happens, http://prntscr.com/8ccm8s

    • Chris Hawkins

      Correct, go to /pihole to see the admin interface 🙂

  • Charles McBeth

    I installed this on a headless Ubuntu server running on a VM using the manual install instructions and it is working great.

    • jacobsalmela

      Excellent! Did you have to modify any commands for Ubuntu?

      • Charles McBeth

        No, I didn’t have to modify anything. Curl was already installed as well. Like the other user installing it on debian, I had to ad the PI user. The only issue I’m having is with the blank web page. It doesn’t seem to be displaying.

        • jacobsalmela

          Does it error out or something?

  • NaaN

    Sorry for the noob question, but what is the URL for the web interface? (Using Raspbian + pihole)

  • Leon Crane

    Seem to have the issue of the admin console not displaying any data

    • jacobsalmela

      Run dietpi-update to update to V88. It seems to have fixed the issue for most users.

      • Leon Crane

        Thanks! that sorted it! its now running fine can see statistics in the admin console.

        One other thing, what command / Script you running to display your statistics on the adafruit screen? am i able to run it on a normal display that pi is plugged into, or can it be run over ssh? if you could give me basic instructions how to do it would be much appreciated, im fairly new to linux so apologise if what im asking seems basic.

        • jacobsalmela

          You can do it over ssh, on the screen, or both.

          1. Install figlet and/or toilet, as well as bc.
          2. Enable the Pi user to log in automatically
          3. Add this script to /etc/profile or the pi user’s .bashrc file

          • Leon Crane

            Can someone break this down into simple steps how to do this please.

          • jacobsalmela

            1. Enter sudo apt-get -y install figlet toilet bc
            2. Follow the instructions in the link mentioned previously to log in the pi user automatically
            3. Paste the script from number 3 at the end of the ~/.bashrc file
            4. Restart

          • Leon Crane

            Cheers Jacob!
            I managed to get it to work, thanks for them instructions.
            I had to ad the script to /etc/profile.
            When i tried adding it to the ~/.bashrc file it did not work.

            Also had to modify the code, as the ip address was not displayed, i then realised it was trying to read from eth0, however i was using wireless adaptor so changed it to wlan0. maybe worth modifying your code so that if it cannot show one then the other.

            and just wanted to say what a fantastic job you have done with all this, i think its great!

          • jacobsalmela

            Awesome! Glad you got it to work. Did you decide to use the 2.4″ screen? I sure had fun with it.

            At some point, I’ll revisit the script and adjust it. I’m focusing on the Web interface now.

          • Leon Crane

            no i just hooked it up to a small hdmi monitor i had, and also used ssh on another computer and run script in that to.
            Web interface is looking good, what parts are you working on next with it?

  • astrolabos

    Hi, I followed the steps of the automated installation one by one and I have an issue. I cannot whitelist sites at all! While I see that the sites excist in Latentwhitelist in /etc/pihole, I cannot see ads displayed in those sites. In one case a site remains blocked even though I have put it in the whitelist.txt in /ect/pihole.
    Thanks in advance for your time

    • jacobsalmela

      Did you run /usr/local/bin/gravity.sh again? You need to do that to apply the whitelist.

      • astrolabos

        Thank you for your immediate answer. Yes I run gravity.sh again and then rebooted but still nothing happened.

        • jacobsalmela

          It’s possible the ads being whitelisted are being served from other domains. Right click and see what domain they show up as.

  • Carsten Overgaard

    I have tried to install the IMG file and the
    DietPi version. After all the installations is done and the Pi boots up it writes:
    FAIL Startpar service(s) returned failure: dnsmasq … failed.

    I have updated everything but still the same error. After a while, I found that
    I could use this command: /etc/init.d/dnsmasq restart but I only works for a couple
    of hours.

    Anyone have and ide, what could be the problem?

    The program is amazing when it work, so hopeful I can find a more stable
    solution.

    Running on Raspberry Pi 2 model B

    • jacobsalmela

      I’m still trying to replicate the issue. Are you running the latest version of DietPi?

      • Carsten Overgaard

        I have downloaded the latest IMG from your site, and upgraded everything. This morning everything didn’t work. Tried /etc/init.d/dnsmasq restart with no succes.

        • Carsten Overgaard

          Dont no if this helps?

          This is from the startup sequence:

          (….) Starting DNS forwarder and DHCP server: dnsmasqchown: invalid user: ‘dnsmasq:nogroup’
          failed!
          .
          ..

          (warn) Kernel lacks cgroup or memory controller not available, not starting cgroup. … (warning)

          • jacobsalmela

            I contacted the author of DietPi to see what the problem might be. I’ll let you know.

      • bleze

        I’m having the exact same problem after a reboot. Updated to 88. dnsmasq fails to start…

        • bleze

          Reinstalled from scratch and running v89 now. Same problem.

          • jacobsalmela

            The DietPi image is a fork of my project so it’s not the exact same as using the automated install command. I contacted the author of DietPi to see if he knows what is causing the error.

          • bleze

            I used the preconfigured image for the Pi and show it patching all the way up to v89. After some restarts I get the described error and adds are not blocked. Funny thing is that it used to work before a reboot when I was on v88…

          • jacobsalmela

            I’m sure it’s something with DietPi as I never get any questions or issues with the automated installer. The DietPi image also uses a forked version of the Pi-hole, so that makes things a little difficult as well. I’ll let you know what the author says.

    • jacobsalmela

      Please run dietpi-bugreport. Follow the on-screen instructions. This will help determine what is causing the error.

      Also,what SD card are you using?

      • Carsten Overgaard

        I ran the bug report 2 days ago, but i haven’t received any confirmation mail. I’m using a Kingston 16 GB Micro SD HC

        • jacobsalmela

          I’m sure he will respond–he’s pretty good at that. Let me know if you find out anything.

          • Carsten Overgaard

            Posted one again, same error with version 90. Did you have a guide to your version that workes 100%

          • jacobsalmela

            Yes, this article that you are posting on will work. It takes a little longer to set up but worth it.

            The DietPi version runs a forked Pi-hole and is different than mine (but uses the same technology). If you want it to work 100% of the time, just follow the instructions on this post.

  • JE

    Hello. When running sudo gravity.sh, I do receive an error message : ‘Fork : insufficient memory”. This is the first time and the Pi Hole v2 is running faithfully for month now.

    • jacobsalmela

      You need to expand your file system and also decrease the memory split to allow less memory to the GUI. Finally, you can increase the swap space.

  • philmiami

    I did the DietPi.img and it “appears” to be ok
    BUT
    there is no gravity.sh file
    do I need to make one?
    If you could please explain all that is needed for dietpi
    It does stop stuff on HTTP sites, but HTTPS I need web browser plugins and also on my iPad/iPhone on the free apps, the ads still appear
    Will this in the end block that?
    Where under DietPi can I see what is blocked, what I can WHITELIST and what file I can add url’s to?
    Thanks
    philmiami

    • philmiami

      I have been conversing with people on the AdTrap forum about this.
      Many are giving up on the Adtrap since it uses Privoxy, must load the entire webpage and then strip out ads.
      This with the DNS is better

      • jacobsalmela

        Yes, and the AdTrap is only 10/100 if I recall so that doesn’t help things.

        One thing I like about the AdTrap is that is can hide video ads, which the Pi-hole still needs some work to do…

    • jacobsalmela

      The gravity.sh file should be there at /usr/local/bin/gravity.sh.

      You can whitelist domains by adding one per line in /etc/pihole/whitelist.txt and then run /usr/local/bin/gravity.sh again to apply it.

      You can see what is being blocked, on both the vanilla Pi-hole and DietPi by tail -fing /var/log/pihole.log.

      I’m working on a troubleshooting/setup guide, which will be on pi-hole.net, but it’s still a work in progress.

      • philmiami

        ok the adtrap is 10/100/1000
        just wanted to answer that from a previous one
        but pihole BEATS IT
        I like what the pihole has
        and I got the gravity thing
        I have been having a good time with tail /usr/pihole.log
        I am getting it on the dietpi
        the adtrap WAS the thing but now what you have done is sooooo much better

        • jacobsalmela

          And more affordable! ?

          • philmiami

            it’s just GREAT blocking and blocking
            my little Pi2 CanaKit from Amazon
            just sits here and sends the ADS TO A BLOCK HOLE………
            SO much better than AdTrap
            There are 2 other people that post in the Adtrap forums, and because one guy posted that he “gave up” on Adtrap and went to the Pi2 and it works, I finally got to mine. It had been sitting here for over a month, and I just let the Adtrap work/not work/unplugged and thrown under the desk……so I did the DietPi Pihole and everything is great.

          • jacobsalmela

            I’m glad it works for you! Now if I could only make $149 on mine like the AdTrap people do!

  • RF

    I have been running pi-hole for a few days and looking at the web interface, it seems to be resetting statistics on GMT time not PDT time as I have the timezone set to.

    root@dietpi:/var/log# date

    Sun Sep 13 19:54:35 PDT 2015

    As you can see my timezone is set to PDT.

    Any ideas on how to fix this?

    Thanks

    • jacobsalmela

      The logs in the interface are shown via some PHP code, but I don’t think that is the issue. Are you using the DietPi image or the automated install command? If it’s DietPi, it is probably something to do with that distro. I’ll ask the author to see if I can find out.

      • RF

        This is the start of the log for tonight, does the log get reset for some reason when the gravity.sh script is run? This is 4 hours later than it restarted two days ago.

        Sep 15 19:17:07 dnsmasq[16126]: exiting on receipt of SIGTERM

        Sep 15 19:17:09 dnsmasq[18258]: started, version 2.62 cachesize 10000

        Sep 15 19:17:09 dnsmasq[18258]: compile time options: IPv6 GNU-getopt DBus i18n IDN DHCP DHCPv6 no-Lua TFTP conntrack

        Sep 15 19:17:09 dnsmasq[18258]: using nameserver 192.168.xxx.10#53 for domain xxxxxxxx.com

        Sep 15 19:17:09 dnsmasq[18258]: using nameserver 192.168.xxx.10#53 for domain xxxx.local

        Sep 15 19:17:09 dnsmasq[18258]: using nameserver 8.8.4.4#53

        Sep 15 19:17:09 dnsmasq[18258]: using nameserver 8.8.8.8#53

        Sep 15 19:17:09 dnsmasq[18258]: read /etc/hosts – 2 addresses

        Sep 15 19:18:17 dnsmasq[18258]: read /etc/pihole/gravity.list – 1613750 addresses

        Thanks.

        • jacobsalmela

          So some people have been having problems with dnsmasq restarting randomly in DietPi. I think the best COA would be to run dietpi-bugreport. The author told me to pass this along to anyone having issues.

          • RF

            I have submitted the bug report, I will update this if I get anything back from the dietpi site.

  • Nameless

    I use DietPi, but the web interface does not update. It sais 0 on everythig except the number of websites blocked.

    • jacobsalmela

      Run dietpi-update to make sure you are at V88 or higher.

      • Nameless

        I am on version 90.

        • jacobsalmela

          Excellent!

  • overzeetop

    Okay, so who’s the joker that added the entire microsoft.com domain space to the blocked list? 😉

    As an aside, is there anyone curating a whitelist of domains for re-enabling many of the desirable click-through trackers (like shopping discount sites, google search page text ad links, and the above mentioned legitimate microsoft links)

    BTW – thanks for doing this. It’s the first valuable use I’ve gotten for my pi.

    • I noticed the same two domains being blocked right after I started using Pi-Hole and was wondering the same thing. Makes me wonder what other domains might be in there that I don’t really want to be blocked! That’s the problem with using third-party curated lists I guess.

      • jacobsalmela

        Ha ha. Yeah, most of the issues with legitimate domains being added come from the mahakala list. Other domains like xkcd.com have been in there, too. You can easily fix this by adding the domains to the whitelist so they don’t get re-blocked when the script runs again.

        I have been thinking of adding a basic whitelist for domains like this that really shouldn’t end up on the block list.

  • Tiemen

    It wasn’t me but good to hear it was blocked (I hope by accident), I’ve putted microsoft.com on the whitelist and it works like a charm

    • Yeah, the mahakala list is the usual suspect for most weird issues. It has the most domains out of any, but also the most issues.

  • jacobsalmela

    You need to use just the domain name, not the https:// part. So it should look like this:

    case $domain in

    "easylist-downloads.adblockplus.org") data=$(....)

    not

    case $domain in

    "https://easylist-downloads.adblockplus.org/malwaredomains_full.txt") data=$(....)

    • hari-bo

      I’ve changed this but recieve the following error

      /usr/local/bin/gravity.sh: line 93: syntax error near unexpected token )'
      /usr/local/bin/gravity.sh: line 93:
      “easylist-downloads.adblockplus.org/malwaredomains_full.txt”) data=$(curl -s -A ‘Mozilla/5.0 (X11; Linux x86_64; rv:30.0) Gecko/20100101 Firefox/30.0’ -e http://forum.xda-developers.com/ -z $saveLocation.”$justDomainsExtension” “${sources[$i]}”);;’

    • hari-bo

      I’ve changed it but the script wont progress anymore.

      ** Neutrino emissions detected…
      Getting adaway.org list… Done.
      Getting adblock.gjtech.net list… Done.
      Getting adblock.mahakala.is list…

      • jacobsalmela

        If you run your script like this, you can see why/where it is failing:

        bash -x /usr/local/bin/gravity.sh

      • azake

        Have you finally managed to get it working?
        I am still stuck at parsing easylist in a proper format.
        I think I have to face the reality. I am not smart enough to get it done 🙁

        • jacobsalmela

          Nope. This is a big task.

          • azake

            I have probably found some useful sed instructions for the easylists, but it seems these are custom sed commands.

            credits to a russian website.

            s/r//g;
            /Adblock/d;
            /.*$.*/d;
            /n/d;
            /.*#.*/d;
            /@@.*/d;
            /^!.*/d;
            /^[.*]$/d;
            s#http://#||#g;
            s////||/g
            s,[+.?&/|],\&,g;
            s/[/\[/g
            s/]/\]/g
            s#*#.*#g;
            s,$.*$,,g;
            s/\|\|(.*)^(.*)/.1\/2/g;
            s/\|\|(.*)/.1/g;
            /^.*$/d;
            /^$/d;

  • Fastfuel

    Hi Jacob,
    Great work and detailed explanations. Just to mention that the “manual” part of the article is a bit confusing for the dnsmasq.conf where it missed the “addn-hosts=/etc/pihole/gravity.list” and you wrote at the beginning “For simplicity, I designed the Pi-hole to use /etc/hosts” that is no more right in the last version of the gravity.sh script.
    A question by the way, how can I add the webinterface to an existing manual conf ? Do i need to create a virtual host ?
    Thank you very much for your work (Hawking would love the astronomical references in the script 🙂

    • jacobsalmela

      I edited the page to address that. I originally used /etc/hosts and later changed it–just forgot to update the article.

      You can add the web interface to the manual setup by running these commands (which I will eventually add to my Hawking-esque script):


      # Download the interface
      sudo wget https://github.com/jacobsalmela/AdminLTE/archive/master.zip -O /var/www/master.zip
      # Unzip it
      sudo unzip /var/www/master.zip -d /var/www/
      # Move it into a folder called admin (or wherever else you want it)
      sudo mv /var/www/AdminLTE-master /var/www/admin
      # Remove the zip file
      sudo rm /var/www/master.zip 2>/dev/null

      Please note that there is a problem accessing the Web interface if you are using the Pi-hole because it redirects everything to the black hole. This is an issue I am still working on. You can, however, access the interface from a device that is not using the Pi-hole, but is on the same network as the Pi.

      • Fastfuel

        thanks. playing around with port and redirections

  • philmiami

    Hey, things are going great with the dietpi version EXCEPT this
    Does this block IPv6?
    All my devices are Apple, and all are or were using IPv6
    I have a Asus RT68U router and have put the IP (static of the Pi2) as the DNS on the LAN settings
    So on my iMac it shows under Network>Ethernet>Advanced>DNS>192.168.0.173 (pi2) and 192.168.0.1 (Router which also has IPv6 enabled)
    It’s just that now the IPv6 DNS’es don’t show in the Advanced>DNS tab
    I could ad them if I wanted to (Google IPv6 DNS servers)
    but
    If I use IPv6 I don’t think the ads are going to be blocked
    tell me what you think

    • jacobsalmela

      If you tail -f /var/log/pihole.log on an IPv6 network, you will see that IPv6 ads still get through. I have had success blocking them, but do not have a detailed post about it. A high-level overview would be:
      1. Enable IPv6 on the Pi
      2. Enable IPv6 for lighttpd and add an additional url.rewrite
      3. Modify /usr/local/bin/gravity.sh to add both an IPv4 and IPv6 address so it adds them both to the hosts file it creates

      • philmiami

        how would I enable IPv6 on the Pi2 and all this?
        I don’t know how to do that

        • jacobsalmela

          It’s too in depth to explain in a comment. You can find plenty of resources online. At some point, I will make a post about it.

  • Mike Saunders

    Thanks for the great guide! I’m trying to use a spankin’- new Adafruit LCD and it’s giving me “cannot load font small” when trying to run the chronometer.sh. Any ideas? I’m using the quick install Raspbian dd image (2015-05-05-raspbian-wheezy-pitft28r_150909) from Adafruit as recommended.

    • Mike Saunders

      NM – I got it. I needed to install figlet (apt-get install figlet) to get this.

      • jacobsalmela

        This should be fixed with an upcoming pull request.

  • philmiami

    I added everything you have for ad domains
    got this:
    ** Aggregating list of domains…
    ** Blacklisting 2 domain(s)…
    ** Whitelisting 16 domain(s)…
    ** 2535472 domains being pulled in by gravity…

    That’s over 2.5 MILLION domains on the pinhole list….
    will see what it breaks…
    the original is doing just fin I just wanted to see to what extreme I could take it and still have a functioning LAN/WAN

    • jacobsalmela

      I haven’t been able to replicate that many domains. I still get 1.6 million. Is there a different source you have–or is a list added twice?

  • jacobsalmela

    You added many more sources than the original script contains. Also, if you did not properly parse them, there may be more entries that are not actually domains.

    • philmiami

      That’s OK with me
      I have let it run for a few days
      Still doing what it is supposed to do except on https web pages
      I still get ads on my iPad even with iOS 9 and now the content blockers from the safari extensions/app store
      Yahoo has gotten quite good at shoving those ads down your throat with their https://www.yahoo.com page
      Seems like nothing can stop them unless you use a full blown laptop/desktop browser

      • jacobsalmela

        Yeah, Yahoo is big on their ads. I use a layered approach: the Pi-hole blocks at the DNS level, AdBlockPlus blocks at the application level using CSS-injection.

        • philmiami

          I have deleted some out because something wasn’t right

          and not

          NOW IT’s3+ MILLION domains
          ** 3134650 domains being pulled in by gravity…

          so whatever

          I also have tried to get the PiGlow LED board (I guess it uses python) to work on the DietPi but it didn’t

          after I did the get this got that from the terminal line

          I want the PiGlow to work

          to pulse with CPU activity (I also got a clear case too so I can see it)

          This is with a Pi 2, not the original Pi

          anyway

          stuck on aggregating
          ——————-

          root@dietpi:~# gravity.sh

          ** Neutrino emissions detected…

          Skipping adaway.org list because it does not have any new entries…

          Skipping adblock.gjtech.net list because it does not have any new entries…

          Skipping adblock.mahakala.is list because it does not have any new entries…

          Skipping hosts-file.net list because it does not have any new entries…

          Skipping http://www.malwaredomainlist.com list because it does not have any new entries…

          Skipping pgl.yoyo.org list because it does not have any new entries…

          Skipping someonewhocares.org list because it does not have any new entries…

          Skipping winhelp2002.mvps.org list because it does not have any new entries…

          Getting easylist-downloads.adblockplus.org list…

          Skipping optimate.dl.sourceforge.net list because it does not have any new entries…

          Skipping tcpdiag.dl.sourceforge.net list because it does not have any new entries…

          Skipping s3.amazonaws.com list because it does not have any new entries…

          Skipping s3.amazonaws.com list because it does not have any new entries…

          Getting s3.amazonaws.com list…

          Skipping tcpdiag.dl.sourceforge.net list because it does not have any new entries…

          Skipping optimate.dl.sourceforge.net list because it does not have any new entries…

          Getting mirror1.malwaredomains.com list…

          Getting easylist-downloads.adblockplus.org list…

          Getting easylist-downloads.adblockplus.org list…

          Getting easylist-downloads.adblockplus.org list…

          Getting easylist-downloads.adblockplus.org list…

          ** Aggregating list of domains…

          ** Blacklisting 2 domain(s)…

          ** Whitelisting 18 domain(s).

          • philmiami

            This weekend I will flash my microSD card (a 32GB one) and try to do the full install instead of the DietPi
            maybe then I can get my PiGlow to work
            I watched a vid on youtube and put it on the first pins like it would be on a original Pi
            something to do this weekend…….

            and maybe try to trim it down from 3.1 MILLION domains
            if that is correct.
            You said to “parse” it?
            How?

          • philmiami

            ** 3134650 domains being pulled in by gravity…

            ** 2079740 unique domains trapped in the event horizon.

            ** Formatting domains into a HOSTS file…

          • jacobsalmela

            You will need to know how to script. Each site will likely be different. I gave some basic instructions in this post on how to do it.

  • Zack

    So I got this up and working pretty easily, kudos to the developer.

    The only problem I have run into is the fact that Windows (I am on v10) does not seem to properly prioritize the DNS servers that are defined in my router. I had them set in my router like this:
    Primary: pi-hole IP
    Secondary: 8.8.8.8
    Tertiary: 8.8.4.4

    It was really a crapshoot whether ads would load, and I think Windows was treating the primary and secondary as equals. My Android phone however always used the primary, so the ads were gone consistently there, which is why I think this is a Windows fault. I’ve since changed the DNS servers as follows in my router:
    Primary: pi-hole IP
    Secondary: pi-hole IP
    Tertiary: 8.8.8.8

    I now no longer have issues in Windows and the ads are always blocked. If for some reason the shit hits the fan and the Raspberry Pi freezes or goes down for any reason, I am hoping Windows will properly switch over to the tertiary and not bring web browsing to a halt. I’d hate for this to happen while I am not home, as explaining to my wife how to fix it over the phone is not going to end well.

    Anyone else notice these Windows DNS server priority issues? Again, hats off to the dev.

    • jacobsalmela

      If you happen to have two Pi’s, you could try setting up two Pi-holes to serve as the primary and secondary, which has worked well for one person I know.

      Personally, I haven’t used Windows 10 yet, so I can’t comment on that. If you have IPv6 on your network, ads still get through that way sometimes. There is a fix for it, just not a documented one.

      • philmiami

        I want to get my IPv6 running like before
        so whenever you get around to documenting that let us know
        or point to directions that someone like myself, let’s say I have to read the directions on how to change a light bulb, would understand

      • Zack

        Yeah I have IPv6 disabled on my router, so I don’t think they’re getting through that way.

        Out of curiosity, what DNS server(s) does Pi-Hole use? Google’s public ones?

        • Konrad Meier

          You probably do not really need those Google entries. I would try and remove them all. Raspbian normally uses the DNS from your ISP, and which are probably faster than Google servers anyway (faster ping). In parts of the planet Google servers are faster though.

          To be more precise, Raspbian probably uses at port 53. The rest is done by the router which knows the ISP’s DNS servers.

    • Chris Hawkins

      I have Windows 10 and it does still prioritize DNS queries in order. To be honest, ive now removed my secondary DNS, but it worked fine with two (piHole primary and google secondary). It only ever fetched pages via googles DNS if the PiHole was down.

      • Zack

        Do you have your DNS settings set within Windows, or is Windows getting them from your router? If I manually put them into Windows then yeah, it followed the prioritization rules, but it does not seem to do so via inheriting them from the router.

      • jacobsalmela

        If you use just the Pi as your DNS server, you can set your secondary and tertiary servers in /etc/dnsmasq.conf. By default, they are using Google’s public servers.

  • Konrad Meier

    When you google for something, sometimes the first search results are on one hand regular results which match the search terms, but on the other hand they are links that are redirected through ad networks. Which results in a blank page when you click on the links.
    Can something be done about this? I noticed that those ad networks unfortunately are some of the big ones, which means that unblocking them would show banner ads all over other websites, too. 🙁

    • jacobsalmela

      Since the Pi-hole works at the DNS level, it will always show a blank page if you go directly to an ad-serving domain that is on the list.

  • Les Girauds

    I’m finding that pihole stops working, or works erratically – sometimes blocking, sometimes not. I find the offending sites are listed on the lists in /etc/pihole so they should be blocked. If I watch the activity on pihole.log, there’s no reference to the site. Can traffic sometimes be routed via pihile and sometimes not? How can I help to diagnose the issue?

    • jacobsalmela

      If you didn’t set a static IP, you might see this behavior. Since the hosts file uses the IP address you had at the time of running /usr/local/bin/gravity.sh, it would break if you were assigned a different DHCP address.

      The other possibility is that some ads still get through via IPv6. The Pi-hole works with IPv6, but I just don’t have an article explaining how to set it up.

      • Les Girauds

        It is possible I didn’t set a static IP address to begin with. I can’t know for your sure now.

        After the rebuild, it didn’t work at all. Configured and recognised on the static IP address but nothing passing through it and no ads blocked. I noted 2 things during the rebuild; a kill -HUP error after gravity.sh had run, and that there was a dietpi version upgrade from v90 to v92.

        I’m going to try again, after removing the MAC address rule I had on the router that assigned an IP address to the RaspPi.

        • jacobsalmela

          Did you restart after updating to V92?

          Check if the Pi-hole is running with this command:

          ps aux | grep -ci [d]nsmasq

          • Les Girauds

            Yes, I restarted.

            That command returns 1

    • Les Girauds

      After the rebuild (double checking my static IP address setting!), I saw “failed to load names from /etc/pihole/gravity.list: No such file or directory” in the pihole.log. I didn’t see if gravity.sh produced any errors during the install process, so I re-ran it.

      That reported ‘no new entries’ for each list so it must have run as a part of the install. Then, after “** Formatting domains into a HOSTS file…” it reported “/usr/local/bin/gravity.sh: line 105: sudo: command not found”. It looks like the install failed somewhere. I manually installed sudo and re-ran gravity.sh. That completed without error.

      On reboot, there were no errors in pihole.log. I changed the DNS on my router to point to the pihole and can see the the log reporting activity, and ads are being blocked. Let the retest commence!

      • jacobsalmela

        The DietPi version uses root as the user instead of pi, so that makes sense that sudo isn’t included by default.

        Glad it’s working again!

        • Les Girauds

          It is working… but I’m experiencing a problem perhaps unrelated and purely coincidental. The router periodically reports the internet being disconnected. I look at the modem and it says there is a connection. 20 – 30s later, the router ‘reconnects’ and all is well again.

          Is it possible that there’s a latency in the DNS (Rasp/DietPi box) that the router is interpreting as a disconnect?

          • jacobsalmela

            You should watch the status page on your router when this happens to see if it is getting cut off there.

            How often does it happen?

          • Les Girauds

            It was happening numerous times a day, 5 or 6 maybe.

            I’ve shelved the Pi-Hole idea for now, until my internet connection becomes more stable/reliable.

          • jacobsalmela

            Sounds like your router might be dying.

          • Les Girauds

            I sincerely hope not! It’s only around a year old and has been very reliable to date. I suspect it could be genuine outages, it happens. I live rurally and am on RADSL – so very variable quality.

  • Øyvind Lauritzen

    I currently use a RPI B version with 256MB RAM, would I benefit with using either the newer B version with 512MB RAM or the brand new with 1GB RAM and a faster CPU for this project? Seems to me sometimes it isn’t as quick to respond as when I did not use a RPI for doing my DNS queries.

    My last option is to maybe use my router to do the job, PFSense doesn’t have a working plugin yet as fas as I understand, but I do run it with ESXI on Intel c2558 chipset and 16GB RAM, is there a easy to understand quide to how I could go forward with that instead?

    • jacobsalmela

      There might be a small benefit, but the Pi doesn’t have to do that much. It just responds to DNS queries and sends a blank HTML file, so there isn’t much processing involved. However, the extra RAM would allow more DNS entries to be cached, thus faster response times.

      The slowness you are seeing is probably a result of the Pi being restarted, which would cause the DNS cache to flush. The longer the Pi-hole is running, the faster it gets because it continues to add domains to it’s cache–up to 10,000 entries.

      I don’t have a B, but running it on my B+ or my Pi2, I don’t notice much of a difference. You could also try getting a faster SD card.

      In all honesty, this project seems a little better for the older models because you can just dedicate it to being a Pi-hole and then use the newer Pi2 for other fun projects.

      As for running this on a different system, as long as you can run dnsmasq and lighttpd, it will work. You may need to make some modifications based on your distro, but I have people running it on many different systems.

    • Konrad Meier

      It’s probably not the Pi but scripts in some websites. The Pi adds just a small amount of milliseconds, usually lower than 10, even when he didn’t cache the DNS response yet. For most probably your router does its own caching, too, and that’s the device which responds to the PI when his cache is empty.

      • Øyvind Lauritzen

        Been using Google’s DNS servers since they were publicly available

        🙂 I think I’ve might solved the problem, it seemed to only occure on my MBP when running OS X, for now cleared the DNS cache and seems to be fine 🙂

      • jacobsalmela

        Have you had better experiences with OpenDNS?

        • Konrad Meier

          No, but it much depends on my location. I suggest users ping each server and see which one replies the fastest.
          My ISP’s DNS server: 4.8 ms
          Google: 6.4 ms (probably running at the ISP)
          OpenDNS: 16.4 ms
          The difference here is negligible, but hundreds of ms can slow things down noticeably.

  • Harald Kroiss

    Hi, i have currently a Problem with the Pi-Hole its very minor but to my (very) anger some people dont like it.

    When i set up the Pi-Hol as DNS server in my WIFI-DHCP configuration everything works good and fine, the only thing is that windows is to stupid to understand that THERE IS an internet connection and shows the yellow triangle at the network symbol. Also the prompt ” you maybe have to login in this network to access internet” is there. My dad has HUGE problems with this symbol and the speech and want this away… (-.-)

    Any idea what is causing this? before Pi-Hole it wasnt there…

    • jacobsalmela

      Are you on Windows 10? Other people were having issues with the DNS priorities. Also, I have noticed there is a slight delay sometimes when you switch to the Pi DNS so it looks like it is disconnected, but then it goes. So if Windows is switching DNS servers, this could be the reason.

      The only other thing I can think is that if you are using the DietPi version, a few people just needed to restart dnsmasq.

      • Harald Kroiss

        Nope happens on Windows 7, strangley on Windows 10 at my PC i have no problem at all. What do you mean with DNS Priorities?

        In my DHCP-Config there is pihole as primary DNS and google-dns at secondary, shouldnt it be this way?

        yeah i am using diepti, Restart dnsmasq on dietpi? When i reboot the pi it doesnt solve the problem.

        • jacobsalmela

          You only need the Pi-hole in your DHCP options as the Pi will offer the clients the other upstream nameservers.

          Try these commands:


          useradd dnsmasq -s /bin/bash -m -g nogroup
          service dnsmasq start

          You can also update to V93, which has fixed many issues.

          • Harald Kroiss

            Thanks, i will try 😉

  • Øyvind Lauritzen

    Regarding your touch screen LCD, is this one of those that goes through SPI or HDMI? I happened to order one with SPI interface, dirt cheap on Aliexpress, will this work? 😉

    • jacobsalmela

      The LCD I was using was from Adafruit and just hooks directly to the GPIO pins. Any monitor will work for the display (although you may need to adjust the font size). It even works over ssh. ?

  • Joan

    Hi, I have just bought a raspberry pi b+ and I’ve set up a VPN server on it. I was wonderig if it would be possible to install Pi-Hole too, so I could improve the speed of navigation through the VPN.

    • jacobsalmela

      Yes. I have a few users who run both so that they can have ad-free browsing while mobile.

  • bigjay517

    Hello,

    I’ve gone through this setup process 2x now, and I can’t seem to get the Pi-Hole to block ads. It is set as my DNS server, and I can confirm that it is resolving my DNS with a:

    dig +trace google.com

    ; <> DiG 9.8.3-P1 <> +trace google.com
    ;; global options: +cmd
    . 1678 IN NS m.root-servers.net.
    . 1678 IN NS e.root-servers.net.
    . 1678 IN NS c.root-servers.net.
    . 1678 IN NS k.root-servers.net.
    . 1678 IN NS g.root-servers.net.
    . 1678 IN NS j.root-servers.net.
    . 1678 IN NS i.root-servers.net.
    . 1678 IN NS l.root-servers.net.
    . 1678 IN NS a.root-servers.net.
    . 1678 IN NS f.root-servers.net.
    . 1678 IN NS b.root-servers.net.
    . 1678 IN NS d.root-servers.net.
    . 1678 IN NS h.root-servers.net.
    ;; Received 228 bytes from 192.168.1.202#53(192.168.1.202) in 43 ms…(truncated rest)

    But when I attempt to ping an entry from the gravity.list, (which should resolve to my Pi’s IP address correct?) I get something other than my Pi’s address.

    ping abchost.us
    ping: cannot resolve abchost.us: Unknown host

    I have done a fresh install of Raspbian, and ran the script and it completes successfully. I have even navigated to the .conf files that it downloads to confirm that they are correct. I have checked my services to verify that dnsmasq and lighttpd are both running.

    I’m a bit lost now. Any ideas?

    • jacobsalmela

      So is your Pi’s IP 192.168.1.202?

      The first thing to check is that /etc/pihole/gravity.list is using that IP address. The two can differ if you are using DHCP or changed addresses another way after running the script.

      The next thing to try to is see that your Pi is actually getting and responding to the requests. On your Pi, run this command:

      tail -f /var/log/pihole.log

      Then, from another device that is using the Pi as it’s DNS server, navigate to or ping a site and see if it shows up in the log.

      • bigjay517

        Hello,

        Yes, 192.168.1.202 is my Pi’s IP address. I have assigned it as static in both the network settings of the Pi and using my routers static DHCP settings.

        I think my gravity.list is not formatted correctly. After reading your response and saying I should see my IP in the list I realized that something was wrong. None of the entries in the list have my IP.

        I have attached a SS of what my file looks like. Shouldn’t there be 2 columns? One with my Pi’s IP and the other with the hostname?

        • jacobsalmela

          Yup. You are missing the IP. If you downloaded the script over the past week or two, it might have been a bug in the script. Just download the latest one and run it again.

          sudo curl -o /usr/local/bin/gravity.sh "https://raw.githubusercontent.com/jacobsalmela/pi-hole/master/gravity.sh"
          sudo chmod 755 /usr/local/bin/gravity.sh

          • bigjay517

            That fixed it for me. It looks like I pulled the gravity.sh script right after the bug was introduced with the IP handling. I just looked at the logs and saw that it was fixed yesterday. Thank you so much! This is working very well so far!

          • jacobsalmela

            Cool. Glad it is working now. Enjoy.

  • Les Girauds

    My router problems aside (& the subject of an ongoing Netgear support conversation), I’ve restarted my PiHole testing. I’ve got to say, it is pretty darned good, so congrats Jacob for some excellent work!

    I have found consistently good results if I change the DNS server on each of my PCs and tablets, simply changing the DNS on my router didn’t seem to be enough BUT, as I mention, I think I’ve got router problems so it could be due to that.

    I have a suggestion. I added a source list to gravity.sh (worked perfectly btw!) and thought it might be useful if the list of source lists could be held as a file separate to gravity.sh. That way, should gravity.sh ever be updated then people wouldn’t need to re-input their sources list.

    This new simplified image installation of dietpi/pihole is a breeze and works perfectly. It makes re-installation, should it be needed, something not to be feared. I really am impressed!

    • jacobsalmela

      Excellent! I’m glad it’s working for you. I’ll definitely look into the separate sources file.

  • Konrad Meier

    Just wanted to add that /etc/network/interfaces is not the method of choice anymore in current Raspbian images. Now, by default, interfaces should be left unchanged, and static IP setup is done via /etc/dhcpcd.conf. If you ignore this and use interfaces, it might give you all kinds of weird trouble. For example, I had 2 ethernet IP addresses – the static one and another DHCP one at the same time (check it with “hostname -I” <- big i).

    • jacobsalmela

      I noticed this when I installed Jessie. I will have to edit the post to reflect this in case people are downloading the latest version. I’m not quite sure how to edit the file yet, but I’m sure there is a guide out there somewhere.

      • Konrad Meier

        I installed the latest Wheezy image and Dnsmasq. After that, /etc/resolv.conf had nameserver 127.0.0.1, permanently. Not sure what happened there.

        • jacobsalmela

          That is strange…

      • azake

        I have a german guide for that. Actually you don’t have to understand the whole part because the most important things are still in english or at list in “unix-language”

        https://www.elektronik-kompendium.de/sites/raspberry-pi/1912151.htm

        This is not my site but the site-owner also mentioned things are changed since Raspbian “Jessie”.

        • jacobsalmela

          Excellent! I will link to that. Thanks.

  • Niels Beukhof

    How do i start that wonderfull realtime monitor on my display?

    • jacobsalmela

      You will need to make sure you have toilet and bc installed. Then save and run this script.

      For best results, you will want to setup your LCD to display the console and automatically log in as the pi user. I have a step-by-step in a comment somewhere on my site. I’ll dig around and see if I can find it. I should really turn that into a how-to page anyway.

      • Niels Beukhof

        hi i tryed to run the script but i get this error:

        bash: line 4: syntax error near unexpected token newline'
        bash: line 4:

        • jacobsalmela

          Did you modify the script? An error like that usually occurs when there is an unterminated quote.

          • Niels Beukhof

            I think i did. Do you mind typing the commands that are needed for installing the script?

          • jacobsalmela

            sudo curl -o /usr/local/bin/gravity.sh "https://raw.githubusercontent.com/jacobsalmela/pi-hole/master/gravity.sh"

          • Niels Beukhof

            i just replace gravity.sh by chronometer.sh?

          • jacobsalmela

            No, those are two different scripts. Just run the above command to replace your gravity.sh.

          • Niels Beukhof

            so that will show info on my tft?

          • jacobsalmela

            Oops. Sorry, I forgot this thread was about the LCD. You can get the LCD script using this command:

            sudo curl -o /usr/local/bin/chronometer.sh "https://raw.githubusercontent.com/jacobsalmela/pi-hole/master/advanced/Scripts/chronometer.sh"

          • Niels Beukhof

            the script works how do i get that nice colerfull asci art pihole logo?

            i get error: error: could not load font small

          • jacobsalmela

            You need to install some additional tools; see this article.

            You may also need to adjust the font size as described here.

          • Niels Beukhof

            ok nice i got it running!! only its really small. It only takes up 1/3 of the screen. Can i adjust this?

          • jacobsalmela

            Yes, using the link provided above (Adafruit).

          • Niels Beukhof

            Perfect now!! Thanks for your help!

          • Niels Beukhof

            And how do i get it to show it direct on startup insted of the console login

  • azake

    Has someone managed to parse correctly the “easylists” ? With my little skills I didn’t manage to parse it correctly. I have tried some sed commands but I didn’t work.
    If you managed it, please be so kind and share your instructions. Thank you.

    • jacobsalmela

      Personally, I have spent considerable time trying this and got no where. I am going to try harder considering so many people are having issue with the mahakala list.

      • azake

        There are some third party approaches to get a clean parse of these lists, but I am not into this to transfer this into piholes script.

  • sandervd

    Bought a Raspberry last week and installed Pi-Hole and it’s great! I was wondering if there would be an easy way to add / remove certain domains on the whitelist.txt via the whitelist.sh script and a cronjob? Say that I want to block Facebook.com on working days from 09:00 till 18:00? Thanks!

    • jacobsalmela

      Yes, that would certainly be possible and pretty straightforward to do (at least in theory). Most people have been putting suggestions/ideas on Github repo. That way, I don’t forget about it! ?

  • Niels Beukhof

    Hey Jacob. How do i exclude website’s from blocking. Im having problems with a few sites not working (for example https://popcorntime.io/) after setting up Pi-hole.

    • jacobsalmela

      You will need to add the desired domains to /etc/pihole/whitelist.txt and then run /usr/local/bin/gravity.sh again. A few things to note: many ads are displayed on sites via a third-party, so even if you whitelist popcortime.io, there may be third-party ads that are still blocked.

      The easiest way to find out what these domains are is to inspect the element via your browser and find the domain the ad is being served from.

      I have a small section on whitelisting in the Pi-hole.net help section (but it still needs some of the details mentioned above).

      • Niels Beukhof

        How do i exacly format the url’s in the whitelist.txt?

        • jacobsalmela

          Just use the FQDN, such as:

          doubleclick.net
          stats.wordpress.com

          One domain per line.

  • Chris Bernhardt

    Jacob, I’m just getting pihole setup but am at a loss. I’m only getting 66,674 domains getting blocked. I’ve ran the gravity.sh multiple times, but it’s not pulling in any more domain to block?

    ** Aggregating list of domains…
    ** Whitelisting 7 ad list sources…
    ** 75582 domains being pulled in by gravity…
    ** 66674 unique domains trapped in the event horizon.
    ** Formatting domains into a HOSTS file…

    • My number of domains dropped from 1.6m to 300k for some reason. I wonder if one of the blacklist sites removed a chunk of domains recently?

      • jacobsalmela

        See the comment above; it was probably due to me commenting out the mahakala list as it was causing issues for many users.

        • Except I haven’t pulled a new version from your repo lately, so the mahakala list is still being grabbed by my gravity.sh script. That’s why I’m confused as to why the large dropped in blocked domains.

          • jacobsalmela

            Hmmm, that is strange. Run this command to see how many domains each list has blocked, which might give some insight:

            sudo find /etc/pihole/ -type f -name list.* -exec wc -l {} ;

            I think I need to add some logic to the script to backup the ad list somewhere since every time you run it, it will overwrite the existing files. That way you would always have something to revert to.

            This happened once before, but the opposite happened: hundreds of thousands of more domains were added to a list. I have some old files with the majority of the domains, but it is from a few months back.

            Let me know if you figure out what happened. My guess is it was the mahakala list as that one seems to be constantly changing.

          • Yep, it’s mahakala all right. It only has 263k domains in it right now.

            263619 /etc/pihole/list.2.adblock.mahakala.is.domains
            659 /etc/pihole/list.1.adblock.gjtech.net.domains
            45218 /etc/pihole/list.3.hosts-file.net.domains
            13734 /etc/pihole/list.7.winhelp2002.mvps.org.domains
            11633 /etc/pihole/list.6.someonewhocares.org.domains
            1488 /etc/pihole/list.4.www.malwaredomainlist.com.domains
            410 /etc/pihole/list.0.adaway.org.domains
            2440 /etc/pihole/list.5.pgl.yoyo.org.domains

    • jacobsalmela

      This was intentional. Due to so many users having legitimate domains be blocked such as microsoft.com, bungie.net, and xkcd.com, I decided to comment out the mahakala list on my repo. Most users report that the ad-blocking performance without the list doesn’t suffer.

      The change also helps new users so they don’t get confused as to why a domain is blocked. And for existing users, they can simply uncomment the list and run /usr/local/bin/gravity.sh again and you will get all those domains back.

  • Pi-Hole was mentioned by Steve Gibson on episode #529 of his Security Now! podcast.

    • jacobsalmela

      This made me smile. Thanks!

    • Dieter Archer

      GO PODCASTING!!!!!

  • Simon Lahaye

    Hi Jacob! Here, have a donation! I ordered a pi just to install pi-hole. I briefly tried it yesterday evening and it’s great. I am now my two daughters’ hero for removing the youtube ads from their ipods. Being a linux noob, I used the Diet-Pi (Wheezy) installation. It went like a breeeze. If I understand correctly, I now have to edit /etc/dnsmasq.conf and/or etc/networkintertfaces in order to use open dns family shield servers ? And also uncomment mahakala and rerun the gravity.sh script to unblock some legit sites?

    • jacobsalmela

      Thank you so much!!

      Yes, just edit /etc/dnsmasq.conf and change the server= values to be the IPs of OpenDNS.

      Actually, the mahakala list is the one that blocks legit sites, so it is turned off by default. Only uncomment it if you want to use the list. In testing, ad-blocking performance was not drastically affected by not using that list.

  • Chris Bernhardt

    How often should the web GUI’s stats be reset? I’ve been running PiHole for about a week now but the web GUI that installed following your instructions doesn’t seem to reset it’s daily statistics.

    • Simon Lahaye

      Hi Chris. My stats would reset in the middle of the day. It sorted itself out once I set the pi’s clock to match my time zone. I use diet-pi and the dietpi-config command was what I used to set my clock. You seem to be having a different issue. Have you tried rebooting?

      • jacobsalmela

        The DietPi version automatically flushes the stats each day. If you used the automated install, then there is a bug where the log file continues to grow, which we are working to fix.

      • Chris Bernhardt

        I have verified that my time zone and system date/time are correct; also rebooted but still the same stats persist on the web GUI.

        • jacobsalmela

          dietpi-update might work, too. The developer made a bunch of fixes since releasing the original image.

  • Olafberzerk

    I installed this on a fresh noobs install 4 weeks ago. Then had to nuke for other reasons. Set it back up again and now I can dig to google.Com from my pi and get the correct ip. When I try an ad domain I get my local ip as intended.

    However when I change my clients to dns to my pi, the resolution times out. Any ideas?

    • jacobsalmela

      The easiest debugging method is to watch the log file:

      tail -f /var/log/pihole.log

      Both lighttpd and dnsmasq need to be running in order for the re-direction to work. I am slowly updating a troubleshooting page, which may also offer some insight.

      • Olafberzerk

        I tried it again with a fresh raspbian jessie install. Verified that lighttpd and dnsmasq are running. On my PI I can dig to the correct IP address is I go to google.com. If I dig to a-ads.com it returns my PI’s address as it should. When I go to a client and set the dns server to my PI’s address and do a nslookup, the DNS server (PI’s address) times out. Here is a tail of the pihole log. I will continue to troubleshoot but any help is appreciated.

        Oct 26 22:36:35 dnsmasq[506]: query[AAAA] mcstats.org from 127.0.0.1

        Oct 26 22:36:35 dnsmasq[506]: cached mcstats.org is NODATA-IPv6

        Oct 26 22:42:25 dnsmasq[506]: query[A] snoop.minecraft.net from 127.0.0.1

        Oct 26 22:42:25 dnsmasq[506]: forwarded snoop.minecraft.net to 8.8.4.4

        Oct 26 22:42:25 dnsmasq[506]: forwarded snoop.minecraft.net to 8.8.8.8

        Oct 26 22:42:25 dnsmasq[506]: reply snoop.minecraft.net is 54.175.97.178

        Oct 26 22:42:25 dnsmasq[506]: reply snoop.minecraft.net is 52.5.111.127

        Oct 26 22:42:25 dnsmasq[506]: query[AAAA] snoop.minecraft.net from 127.0.0.1

        Oct 26 22:42:25 dnsmasq[506]: forwarded snoop.minecraft.net to 8.8.8.8

        Oct 26 22:42:25 dnsmasq[506]: reply snoop.minecraft.net is NODATA-IPv6

        • Olafberzerk

          Oops.. Should have looked at dnsmasq.conf.. Was listening on eth0, when I needed to change to wlan0. All works now! Thanks!

        • jacobsalmela

          Everything seems in order. I just did a fresh install again and I cannot replicate the issue. Maybe it is something to do with IPv6 on your network.

  • philmiami

    I have been away working

    home for a day, so looking as to why this is not working……
    have run dietpi-update
    now on latest (like 98 or 99)
    1st: Do I always have to type GRAVITY.SH to have it run? After a reboot/etc it should run auto
    2nd: I get this AFTER it runs gravity.sh and I would guess completes:
    root@dietpi:~# gravity.sh
    ** Neutrino emissions detected…
    Skipping adaway.org list because it does not have any new entries…
    Skipping adblock.gjtech.net list because it does not have any new entries…
    Skipping adblock.mahakala.is list because it does not have any new entries…
    Skipping hosts-file.net list because it does not have any new entries…
    Skipping http://www.malwaredomainlist.com list because it does not have any new entries…
    Skipping pgl.yoyo.org list because it does not have any new entries…
    Skipping someonewhocares.org list because it does not have any new entries…
    Skipping winhelp2002.mvps.org list because it does not have any new entries…
    ** Aggregating list of domains…
    ** Whitelisting 4 domain(s)…
    ** 339450 domains being pulled in by gravity…
    ** 303354 unique domains trapped in the event horizon.
    ** Formatting domains into a HOSTS file…
    kill: usage: kill [-s sigspec | -n signum | -sigspec] pid | jobspec … or kill -l [sigspec]
    root@dietpi:~#

    *******************
    why does it say KILL? Is the gravity.sh NOT running? I type
    tail -f /var/log/pihole.log
    and get NOTHING…..
    I used to could sit here and watch all network traffic now, nothing
    so don’t know if it’s working or not

    • jacobsalmela

      /usr/local/bin/gravity.sh only needs to run to update the ad-lists and apply any whitelist/blacklist changes. The script will not run automatically (yet) unless you set up a cron job.

      The kill is there to restart dnsmasq so that the changes can be applied.

  • John McCambridge

    The steps to set a static IP are incredibly confusing. Comments that are listed as outdated – comments that are crossed out – steps to “look above”

    Can anyone help me?

  • Denis Demchenko

    I faced up with one fun problem. After installation of Pi-hole on my Rasp I started tests and found that ‘dig’ shows absolutely different result localy on Rasp and on my Win 7 host. Pi-hole worked fine on Rasp but Win 7 still used real DNS records. I double checked my DNS setting on Win 7, disabled IPv6 but it did not help. I dug in dnsmasq documentation bit all looked good on Rasp side. I ran my another notebook with Linux Mint and was surprised that Pi-hole worked for it. It looked like my Win 7 host ignores all DNS querries to Pi-hole and use something else. And I found this ‘else’. I use Avast antivirus and it has SecureDNS option enabled by default. So, in this case it does not matter that I set in your network configuration, Avast did all for me 🙂 I think this information can save some time for someone 🙂

    • jacobsalmela

      Interesting. There have been a few more people who had issues in WIndows with DNS resolution. I wonder if they had Avast…

      Thanks for the tip. I will have to add it to the FAQs.

  • Mike

    Hi Jacob,

    First of all, thanks very much for putting out this amazing resource. I have been adapting your manual instructions to run on Centos 7 server in a VPS.

    There are a number of lighttpd.conf changes required for Centos to get the stats webpage to load (mimetypes, different php commands, etc). I am currently at that point, where the stats page (/admin/index.php) loads and displays stats. However, when I click on “More Info”, expecting the screen to change with more verbose details, nothing really happens.

    What is the current expectation when clicking “More Info” on each of the 4 boxes? I couldn’t find any screenshots to know if it should provide more details or if this is just a placeholder and not yet useful?

    Is the index2.html needed? It seems like its an example file or something and shouldn’t be included? Scared the pants off of me at first until I realized it was old, static data and not some dude’s live website. I’m assuming that guy provided that file as an example…

    Thanks in advance!

    • jacobsalmela

      I’m glad you like it! If you would be willing to share your customizations when they are all done, I would like to provide instructions for multiple operating systems, but I understand if you don’t. =)

      The “More Info” links are just place holders right now. I’m working on a database to store historical statistics, which those links will eventually point to.

      No, index2.html is not needed. The site is a fork of AdminLTE. That page is just an example of everything you can do. I left it there so I can use it as a reference to charts/graphs that I want to add to the interface.

      • Mike

        Thanks for confirming the placeholders for More Info.
        Yes, once I’ve had some cycles on it and know its working pretty good, I’ll test it on a fresh VPS install and then document the process. Unfortunately, I didn’t record all the changes as I went, I just google’d as I encountered problems, so I’ll need to do some testing to recreate all the changes.
        It would be fantastic to have an OS agnostic script, so FreeBSD, Ubuntu, centos, etc can easily have ad blockers with auto updating scripts.

        • jacobsalmela

          I agree. I didn’t think the project would be as big as it is, so I wasn’t thinking beyond the Raspberry Pi. But now people have been putting it on CentOS, Mint, and a very special Sky-Hole on Ubuntu.

  • Zack

    So I’m finding that a lot of the referral links that are on slickdeals get blocked. When I click on something that takes me to the product page, it craps out at the referral URL, and never makes it to the actual URL of the product page. I started whitelisting them as I found them, but when I realized it wasn’t just 2 or 3 domains I gave up. Is there an easier way to whitelist these referral links? Maybe a known whitelist list I can subscribe to?

    • Zack

      So I found a collection of domains to whitelist which is great: http://slickdeals.net/f/6465800-list-of-hosts-to-add-to-adaway-adblock-white-list

      Is there a way to whitelist an entire domain (and all subdomains)? IE if I want to whitelist all subdomains for redirectingat.com, to I have to add them all individually, or is there some tag I can apply to it so it whitelists everything?

      • I believe currently you have to list each domain separately, no wildcards.I believe currently you have to list each domain separately, no wildcards.

      • jacobsalmela

        At this point, each domain needs to be entered, one per line. Wildcards are not supported yet. But since you have the list, you could just copy/paste them into your whitelist. Here is the FAQ on that.

  • Bushi

    Loving pi hole but seeing some errors when trying to run gravity for updates… suggestions?

    pi@raspberrypi ~ $ /usr/local/bin/gravity.sh

    ** Neutrino emissions detected…

    Getting adaway.org list… Skipping pattern because transporter logic detected no changes…

    Getting adblock.gjtech.net list… Skipping pattern because transporter logic detected no changes…

    Getting hosts-file.net list… /usr/local/bin/gravity.sh: line 94: /etc/pihole/list.2.hosts-file.net.domains: Permission denied

    Done.

    Getting http://www.malwaredomainlist.com list… /usr/local/bin/gravity.sh: line 94: /etc/pihole/list.3.www.malwaredomainlist.com.domains: Permission denied

    Done.

    Getting pgl.yoyo.org list… /usr/local/bin/gravity.sh: line 94: /etc/pihole/list.4.pgl.yoyo.org.domains: Permission denied

    Done.

    Getting someonewhocares.org list… /usr/local/bin/gravity.sh: line 94: /etc/pihole/list.5.someonewhocares.org.domains: Permission denied

    Done.

    Getting winhelp2002.mvps.org list… Skipping pattern because transporter logic detected no changes…

    ** Aggregating list of domains…

    /usr/local/bin/gravity.sh: line 106: /etc/pihole/pihole.0.matter.txt: Permission denied

    find: cat' terminated by signal 13

    find: cat’ terminated by signal 13

    find: cat' terminated by signal 13

    find: cat’ terminated by signal 13

    find: cat' terminated by signal 13

    find: cat’ terminated by signal 13

    find: `cat’ terminated by signal 13

    rm: remove write-protected regular file ‘/etc/pihole/latentWhitelist.txt’? y

    rm: cannot remove ‘/etc/pihole/latentWhitelist.txt’: Permission denied

    ** Whitelisting 7 ad list sources…

    /usr/local/bin/gravity.sh: line 159: /etc/pihole/latentWhitelist.txt: Permission denied

    /usr/local/bin/gravity.sh: line 159: /etc/pihole/latentWhitelist.txt: Permission denied

    /usr/local/bin/gravity.sh: line 159: /etc/pihole/latentWhitelist.txt: Permission denied

    /usr/local/bin/gravity.sh: line 159: /etc/pihole/latentWhitelist.txt: Permission denied

    /usr/local/bin/gravity.sh: line 159: /etc/pihole/latentWhitelist.txt: Permission denied

    /usr/local/bin/gravity.sh: line 159: /etc/pihole/latentWhitelist.txt: Permission denied

    /usr/local/bin/gravity.sh: line 159: /etc/pihole/latentWhitelist.txt: Permission denied

    /usr/local/bin/gravity.sh: line 163: /etc/pihole/pihole.1.andLight.txt: Permission denied

    ** 76109 domains being pulled in by gravity…

    cp: cannot create regular file ‘/etc/pihole/pihole.2.supernova.txt’: Permission denied

    /usr/local/bin/gravity.sh: line 126: /etc/pihole/pihole.3.eventHorizon.txt: Permission denied

    ** 67053 unique domains trapped in the event horizon.

    ** Formatting domains into a HOSTS file…

    /usr/local/bin/gravity.sh: line 132: /etc/pihole/pihole.4.accretionDisc.txt: Permission denied

    /usr/local/bin/gravity.sh: line 136: kill: (472) – Operation not permitted

    • jacobsalmela

      It looks like you probably need to use sudo to run the script.

      • Bushi

        That did it, thanks 🙂 !
        pi@raspberrypi ~ $ sudo /usr/local/bin/gravity.sh
        ** Neutrino emissions detected…
        Getting adaway.org list… Skipping pattern because transporter logic detected no changes…
        Getting adblock.gjtech.net list… Skipping pattern because transporter logic detected no changes…
        Getting hosts-file.net list… Done.
        Getting http://www.malwaredomainlist.com list… Done.
        Getting pgl.yoyo.org list… Done.
        Getting someonewhocares.org list… Done.
        Getting winhelp2002.mvps.org list… Skipping pattern because transporter logic detected no changes…
        ** Aggregating list of domains…
        ** Whitelisting 7 ad list sources…
        ** 76190 domains being pulled in by gravity…
        ** 67146 unique domains trapped in the event horizon.
        ** Formatting domains into a HOSTS file…

  • Rodrigo Bozzolo

    Hey Jacob,

    I installed this in my home network and I love it! Thanks for creating it! However, it doesn’t seem to be blocking ads for mobile apps in Android – in particular, YouTube is really damn nasty about its ads and I’d like to sanitize it. Any idea on how to resolve that, if it’s possible?

    • jacobsalmela

      It’s not perfect yet as the Pi-hole relies on third-party lists. One of the best ways to find out what domains are being queried when you are using Youtube or an Android app is to watch the log file and then whitelist the domain(s) you find.

  • Rodrigo Bozzolo

    Oh, one more thing:

    You might want to add new instructions on how to set this up as your network’s DNS server by changing the settings on your router/modem. I have it set up this way and it worked instantly, no need to go device by device.

    • jacobsalmela

      I plan to do this in the FAQ section at some point. This project is getting so large it’s getting hard to keep up!

    • Les Girauds

      I set up my pihole’s IP address in the router some weeks ago but, tbh, it didn’t seem to catch everything. Somehow, traffic was bypassing the pihole – how, I don’t know. I find it much more effective configured at device level.

      That said, I found the chrome browser on my linux desktop displayed adds. Switch to any other browser for the same URL and ads where gone. Chrome maybe doing something clever…

      The Opera browser on my android tablet did similar until I turned off the ‘data saving’ option! I think because, in data saving mode, Opera assembles the web page on their own servers and sends the completed page to the browser.

      • jacobsalmela

        Did you set only the Pi-hole as a DNS server? If you happened to set the secondary and tertiary servers to something else, it might be using that. You basically need to rely completely on the Pi-hole as the standalone DNS server. The Pi-hole is configured to use Google’s DNS servers as the upstream ones.

        • Les Girauds

          Probably not, I probably put in a secondary DNS address too. I’ve since changed my ISP and I’m unable to change DNS settings on the (ISP provided) router. So I have to configure each device separately anyway!

  • Anthony

    Hi
    Pi-Hole seems to be working for the most part. However, when it comes to Yahoo.com it seems to let in all advertising. ADP (AdBlock Plus) stops advertising from Yahoo so I’m a bit lost as why Pi-Hole lets them through. Also, it does not matter if systems are running Linux or Windows (Yahoo ads seen other sites no ads).
    Any help would greatly be appreciated. If you need more info, I’ll provide.

  • Samma

    Hi. Thanks for awesome tutorial. I see dnsmasq comes under GPL license. Can you suggest any other lightweight DNS which comes without GPL ? Thanks

    • jacobsalmela

      Offhand I know of Unbound (BSD license), and BIND (ISC license).

      • Samma

        Is there any other light weight ( for mobile devices ) , just like dnsmasq? I have tried BIND, but it heavily consumes mobile resources.

        • jacobsalmela

          I’m sure there are others, but dnsmasq has been extremely refined over the years so I’m not sure you if anything else will compare.

          • Samma

            Ok. If I find something, I will post here in reply.
            Thanks a lot for tutorial and your quick replies 🙂

          • jacobsalmela

            No problem!

  • Simon Lahaye

    I just got myself a Samsung Smart TV. The ad blocking works perfectly with all its VOD “Smart Hub” apps.

    • jacobsalmela

      Nice! It really does work for all your network devices!

  • Morgan

    Nice guide! I’ve done this to a Ubuntu Server VM on my server at home. However, when I woke up today it seems the whitelist doesn’t work anymore. I’ve redone the whitelist, ran sudo /usr/local/bin/gravity.sh. On my machine I ran ipconfig /flushdns but it still won’t whitelist.
    I also wonder if there are any solutions to streamed media, like here in sweden, I’m using tv3 play to watch tv shows, which shows ads inbetween episodes, but it says “You seem to use an adblocker blabla ..” and I can’t continue watching. I’ve noticed they are using google advert and I don’t want to whitelist that cause then there is no point using this 😛 Is there some kind of workaround?

    • jacobsalmela

      Whitelisting works a bit differently than most ad-blockers; see this FAQ for more help.

      As for the streaming, if you are talking about Hulu, I haveccovered this in depth. I think it will eventually be possible, it will just take some engineering.

      • Morgan

        Thanks for your quick answer. I’m not talking about Hulu but I guess it’s similiar so I will take a look.

        For the whitelisting, I don’t get any more result from the FAQ. I have whitelisted what seems to be required, and it worked same say I installed it, and that evening. But when I woke up the morning after, the whitelisting didn’t work. No reboots or anything was made. I have to manually remove the entries from gravity.list to make them work.

        Gravity script says it whitelisting the domains so I guess the script reads whitelist.txt. Just doesn’t seem to remove them from gravity.list.

        • jacobsalmela

          When did you install the Pi-hole? The whitelist.sh script was updated a while back and works much better now. So if you were using an old one, it might not have worked as well.

          • Morgan

            Yesterday. whitelist.sh is working fine, it’s gravity.sh that isn’t removing the whitelist.txt items from gravity.list.

  • B34N3R

    How do I resolve local hostnames? I have a firewall (LMCE) resolving local names. I put pi-hole address as 1st and firewall as 2nd in ubuntu but no locals

    • jacobsalmela

      I believe you need to remove domain-needed from /etc/dnsmasq.conf. I don’t remember why I put this in there, but please let me know if it works or not. I will turn this into an FAQ.

  • John Leura

    This is a dumb question. Can someone please tell me how to enter the “|” symbol in the command line. I know this is called the pipe sign. Every time I push the shift + key combination it give me a ~. Please help thanks. I am trying to run the script but I can seem to get the pipe symbol

    • jacobsalmela

      You might have a different keyboard layout, which might cause the tilde to appear instead of the pipe. The fast solution is to just copy and paste the pipe ( | ). Otherwise, change your keyboard layout. Finally, if you have a way to enter ASCII codes, it is code 174.

    • Simon Lahaye

      I usually copy from my browser and paste into the putty terminal window I use to control my pi.

    • jacobsalmela

      I might have another solution:

      Edit /etc/default/keyboard, then change xkblayout="gb" to xkblayout="us"

      Then reboot to apply the changes:

      sudo reboot

      I don’t have a machine to verify really, since I have no issues typing a pipe, but I found this solution that might work and thought of you.

  • Andrew

    Hello, Somewhat of a novice at this but I just can’t seem to get this to work. I have raspbian Jessie installed I ran the curl -L install.pi-hole.net | bash command in the terminal and it went through it’s whole setup process and then completed and gave me the IP to use and so I’ve tried manually entering it on a couple of computers as well as just adding it in my router, I’ve flushed my DNS but It just isn’t working so if someone could help me that would be great. Again just let me know if there is some information that you need. I have also looked in the pihole log and I will see /etc/pihole/gravity.list query and my pi’s IP address so I mean it seems like it’s working sometimes but yet I still see ads so maybe I am just being dumb and missing a step(s) but I just don’t know. I also have Avast that’s the antivirus that all my computers run so I saw that avast could cause issues but if so how do you resolve them then.

    Thanks, Andrew

    • Simon Lahaye

      Maybe clear your browser cache?

      • Andrew

        I’ve done that still no luck and I’ve tries multiple browsers chrome Firefox ie

        • Under /etc/pihole do you have a large gravity.list file? Have you tried re-running the gravity script to build the blacklist files?

          • Andrew

            Novice here so would you mind explaining in more depth. Also my explanations might be vague or just not understandable so please bare with me. when I go To a page while the log is open it just moves so fast that I can’t keep up I’ll try to take a video and post a link to it on YouTube

          • I’ve seen the occasional ad make through to my Yahoo homepage as well. I suspect they are coming from a site that Pi-Hole isn’t blocking. The only way to “fix” those would be to inspect the HTML of the page to determine where the ad is coming from and then add it to your blacklist.txt file.

          • Andrew

            Ok how do I create/add to a blacklist file?

          • You just need to create a text file named “blacklist.txt” in your /etc/pihole directory and put in any domains you want to additionally block. You can’t use wildcards, though, so you have to list each domain separately, like:

            ad-server.domain1.com
            ad-server1.domain1.com
            here-have.some-ads.com

            Then re-run your gravity.sh script and Pi-Hole will incorporate your blacklist entries.

            You can also do the same with whitelist.txt to override sites you DON’T want blocked.

          • Andrew

            alright that seemed to help in some cases, but I am still getting the sponsored ads like say in my email right at the top under like select all and sorting options, I have inspected the element and added it to my blacklist but it still shows up. I have also cleared all of my browsing history/cache etc but still shows up. It shows up as beap.gemini.yahoo.com when I inspect it.

          • jacobsalmela

            The Pi-hole still struggles with IPv6 ads and SSL, so it doesn’t block everything yet. I’m a single developer running off of donations, so advancement is a bit slow. There are many people who have been helping though.

            Finally, you may want to check out the FAQs I have been putting together, which may help you figure out why it isn’t working.

  • What are people’s thoughts on the blocking of the Google Analytics? Both http://www.google-analytics.com and ssl.google-analytics.com are blocked by Pi-Hole. I was having an issue with a site not working properly and eventually tracked it down to this. I had to add those two sites to my whitelist.txt. I think I’d prefer to block Google Analytics but was wondering what other people do.

    • Nichlas H.

      I’ve whitelisted Google Analytics in the past, but later I concluded that even though they are not serving any ads, they do amass an enormous amount of data on all webusers, I’ve come to see it as a privacy issue.

      In my mind, if people want stats for their web-pages, they should get it without selling their users out at the same time. As an example, see SpiderOak’s take on it https://spideroak.com/articles/yeah-we-ditched-google

      Also, I’m not sure I would allow analytics just because it breaks one page.

      • I agree. I’m a SpiderOak user too, so I saw that article. 🙂

  • Nichlas H.

    I’ve successfully run the install on my Pi, and so far, i’ve succeeded in using it as a DNS server for my whole network, but no domains are redirected at all.

    When i run gravity it has nothing to update, but my gravity.list file is still empty.

    Looking at the log-file all domains are resolved from the opendns servers, and none from the Pi.

    Is there some way to clean and recreate the gravity list?

    • gravity.list shouldn’t be empty (obviously) … mine is ~2.6mb. I assume you’ve tried just flat out deleting the gravity.list fine and then re-running gravity.sh? What’s the output from gravity.sh look like?

    • jacobsalmela

      There was recently a bug that made gravity.list empty on install. Try downloading the latest script:


      sudo rm /etc/pihole/list.*
      sudo rm /usr/local/bin/gravity.sh
      sudo curl -o /usr/local/bin/gravity.sh https://raw.githubusercontent.com/jacobsalmela/pi-hole/master/gravity.sh

  • hunter

    If you want to use pi-hole with services like Unblock-Us, add these to your /etc/dnsmasq.conf

    ######

    server=/netflix.com/208.122.23.22
    server=/netflix.com/208.122.23.23
    server=/unblock-us.com/208.122.23.22
    server=/unblock-us.com/208.122.23.23
    server=/hulu.com/208.122.23.22
    server=/hulu.com/208.122.23.22
    ######

  • Darren

    Hi,

    Had Pi-Hole installed for a while now on my home Pi1 general server to cover my home network, and so far it’s been doing fine. However an access problem has come up that I can’t seem to work around.

    Specifically my wife needs access to a certain site (https://rmsurveys.research-int.com/) which Pi-Hole seems to object to. I’ve tried adding it to a whitelist.txt file, which seems to get accepted (when I re-run gravity.sh it lists 1 whitelisted domain), but I still can’t get access to the site. It either lists as connection refused or just comes up with a white screen.
    If I flip my tablet over to a static IP address and use a direct Google DNS (8.8.8.8) then I can access it fine, but if I go via the Pi-Hole it fails again. I’ve tried setting the Pi-Hole DNS from OpenDNS (which I usually use) to Google and that didn’t help, so I’m at a bit of a loss as to what to do next to get normal access.
    Can anyone give me some advice on what to try, as whilst I can work around by using direct DNS I’d prefer to have a proper solution if possible.

    • jacobsalmela

      Try using the whitelist.sh script and see if that does it. gravity.sh does it slightly differently. I’m thinking of using the whitelist script permanently if this solution works for you. Please let me know.

      • Darren

        Thanks for the quick reply. Where do I find that script, it doesn’t seem to be anywhere in my installation? I installed via the automated method a while ago, so I guess I may be on an older version of PiHole?

        Do I need to update somehow? If so how?

        • jacobsalmela

          It has been changing a lot lately, and I don’t have a great update mechanism yet. You can download it with this command:

          sudo curl -o /usr/local/bin/whitelist.sh https://raw.githubusercontent.com/jacobsalmela/pi-hole/master/advanced/Scripts/whitelist.sh

          • Darren

            Thanks, unfortunately it doesn’t seem to be working. I tried it, got the result below and now I get a 404 error when I try and reach the site.

            pi@PiVPN /usr/local/bin $ whitelist.sh http://rmpanelzone.research-int.com/

            Whitelisting http://rmpanelzone.research-int.com/

            /usr/local/bin/whitelist.sh: line 38: /etc/pihole/whitelist.txt: Permission denied

            Modifying hosts file…

            sed: -e expression #1, char 36: unknown option to `s’

            sed: -e expression #1, char 33: Unmatched ( or (

            ** 1 domain(s) whitelisted.

          • Darren

            Ok tried it. The script ran, but I’m now getting “404 – Not Found” when I try to access the site.

            This was by running “sudo whitelist.sh rmpanelzone.research-int.com” from the command line, which reported back 1 domain(s) whitelisted.

            Am I doing something wrong, or do I need to include a https:// or something?

          • Darren

            Hmm, tried again and now it’s working? So I’ll test it longer-term and check it keeps workign, but it’s looking promising now.

            Thanks and sorry to cause so much trouble

          • jacobsalmela

            Yes, you need to use just the domain name, not the http:// or the rest of the URL.

  • Aaradhya Singh

    really love you you can read here : https://www.robomart.com/buy-raspberry-pi-2-india-price

  • JE

    Hi Jacob,

    Suggestion: you could add ‘namebench’ to the install script. It has nice functions like testing the speed of a local dns vs global dns providers and it can load the top 25.000 dns sites which speeds up loading after the initial installation or reboot I think.

    I reinstalled yesterday on a Raspberry 2.0. If I compare the initial installation procedure with the current procedure, I can only be impressed.

    Thanks for all the hard work.

    George

    • jacobsalmela

      I will definitely look into that. Thanks!

  • Andrew

    Hello,I have an issue I just Re ran the Gravity.sh for updates and it found some but now all of a sudden I am getting ads on youtube videos where I previously was not before. So what happened and how can I fix this.

    Thanks,

  • Gar

    Hi, I’ve been trying to ‘roll my own’ on an Ubuntu 14.04.3 LTS Virtual machine.. dnsmasq /gravity is working , but lighttpd is barfing with

    pi@pihole:~$ sudo service lighttpd restart
    2015-12-21 13:14:26: (configfile.c.589) source: /etc/lighttpd/lighttpd.conf line: 32 pos: 25 use => for assignments in arrays

    when i use the conf file detailed in the ‘build from scratch’, detailed above..

    What am i missing.. is the conf file above been ‘re-coded’ by the hosting website?

    • jacobsalmela

      I have successfully built a Pi-hole in an Ubuntu VM.

      I think some of the formatting in my code on this post did get messed up, so if you copy/pasted it, that was probably the issue. I will try to get that fixed when I can.

  • carl

    hope to see the virtual machine version, it would be nice for some one do not have Raspberry Pi . Thanks

  • jacobsalmela

    Pi-hole was mentioned on TekThing:

    https://youtu.be/8Co59HU2gY0?t=2m

  • Andres Torrubia

    Setting piholeIP=0.0.0.0 removes the need of the web server altogether. Im using it w/o web server on an OpenWRT and no issues so far. Any reason you didn’t take this (simpler) route?

  • Daniel Čavničar

    Hi. I’m using Pi-Hole and it works perefectly. I’m using it with dnsmasq and web server to serve blank page.

    Now i have a little problem. I want to redirect acess to certain IP/website to an IP in my local network. How do i accomplish that? Thank you

    • jacobsalmela

      Beat me to it.

      address=/domain.com/1.2.3.4

      If you are still using adList.conf, you have the original Pi-hole. I’m glad it’s still working well after all this time.

      • Daniel Čavničar

        It’s a great thing since my main router Mikrotik doesn’t have any option to block ads 🙂

      • Daniel Čavničar

        Ok. Now i installed new version of Raspbian and new version of PiHole using the script above (Working great). How can i redirect DNS request to different IP in this version of Pihole?

  • mattmill98

    Is a 8GB SD card sufficent? As i see it stores ‘lists’ on the pi. How much storage do the lists require?

    • jacobsalmela

      Yes, an 8GB card is plenty.

  • Nick Tuinman

    Thanks! It’s running great, even doubled the blacklist domains with my own sources.

    However, I am encountering a strange problem. When I point my hosts to pi-hole statically it runs fine. But if I change my router dhcp settings to automate this, the pi-hole is not able to resolve any hostname.

    At first I thought I had accidentally created a loop since pi-hole looks to itself for dns;

    ;; ANSWER SECTION:
    google.com. 194 IN A 216.58.209.174
    ;; Query time: 39 msec
    ;; SERVER: 127.0.0.1#53(127.0.0.1)

    So I changed that and verified (I’m using Comodo’s pre-filtered dns, just for kicks);

    ;; ANSWER SECTION:
    google.com. 60 IN A 216.58.209.174
    ;; Query time: 26 msec
    ;; SERVER: 8.20.247.20#53(8.20.247.20)

    But even then every time I change the DHCP settings on my router it prevents pi-hole from resolving everything.

    Am I overlooking something?

    • jacobsalmela

      It’s a bit confusing. You should also know that some routers just don’t seem to work with it.

      Are you certain you are entering the Pi’s IP address in the correct spot? It should also be the only DNS server, even if there are slots for two more. The Pi-hole has upstream servers set in /etc/dnsmasq.conf.

      You can also take a look at this FAQ.

      • Nick Tuinman

        Quite sure! my router speaks unix, so configuring is easy. the Pi is the only one configured in DHCp.

        It’s just so odd. For troubleshooting purposes only the pi, 1 host and the router are in play.
        I hardcoded static IP settings for the pi, including the upstreams and disabled dhcpcd completely. I verified which upstreams are requested from the Pi after. Again, when statically pointing the host to the pihole, it works like a charm. When changing the DHCP server on the router, the pi immediately stops resolving. All upstreams remain pingable from the Pi, but it just wont resolve.

        • jacobsalmela

          What kind of router do you have? Can you post a screenshot of where you are making the setting change?

        • jacobsalmela

          What kind of router do you have? Can you post a screenshot of where you are making the setting change?

          • Nick Tuinman

            An Asus RT-AC56U running asuswrt-merlin
            http://asuswrt.lostrealm.ca/features

            See output here;

            ghost@RT-AC56U:/# cat /etc/dnsmasq.conf
            pid-file=/var/run/dnsmasq.pid
            user=nobody
            bind-dynamic
            interface=br0
            interface=ppp1*
            no-dhcp-interface=ppp1*
            resolv-file=/tmp/resolv.conf
            servers-file=/tmp/resolv.dnsmasq
            no-poll
            no-negcache
            cache-size=1500
            min-port=4096
            dhcp-range=lan,192.168.1.2,192.168.1.250,255.255.255.0,86400s
            dhcp-option=lan,3,192.168.1.252
            dhcp-option=lan,252,”n”
            dhcp-authoritative
            read-ethers
            addn-hosts=/etc/hosts.dnsmasq

            ghost@RT-AC56U:/# cat /tmp/resolv.conf
            nameserver 192.168.1.252

            ghost@RT-AC56U:/# cat /etc/hosts.dnsmasq
            192.168.1.252 pihole

            Which translates to the webpage here I suppose, rarely use it though;
            https://i.imgur.com/kPyjj8l.png

          • jacobsalmela

            Hmm, yeah everything looks in order. What about some other utilities like dig or traceroute to find out a bit more?

  • Matt Brotherson

    Howdy. Installed pi-hole today and ads are successfully being blocked. One small issue — when issuing an nslookup command from windows, it resolves my pi-hole DNS server name to the first entry in gravity.list. Additionally, network discovery tools report my pi hostname now as the first entry in gravity.list as well. (I’ve verified by removing the IPv4/IPv6 entries for the first host in the list and rebooted the pi. This results in the hostname being reported as what was the second host in the unedited file, which is now the first entry.) The pi itself responds to a hostname command with the correct hostname.

    It’s a minor issue but kind of comical when my network discovery tools report my pi as one of the ad hosts. It does not affect the pi-hole functionality though. Any ideas? (A quick hack was to add the hostname I wanted for the rPi to the first line of gravity.list, but I suspect this file is automatically updated.)

    Thanks for your work!

  • Matt Brotherson

    Howdy. Installed pi-hole today and ads are successfully being blocked. One small issue — when issuing an nslookup command from windows, it resolves my pi-hole DNS server name to the first entry in gravity.list. Additionally, network discovery tools report my pi hostname now as the first entry in gravity.list as well. (I’ve verified by removing the IPv4/IPv6 entries for the first host in the list and rebooted the pi. This results in the hostname being reported as what was the second host in the unedited file, which is now the first entry.) The pi itself responds to a hostname command with the correct hostname.

    It’s a minor issue but kind of comical when my network discovery tools report my pi as one of the ad hosts. It does not affect the pi-hole functionality though. Any ideas? (A quick hack was to add the hostname I wanted for the rPi to the first line of gravity.list, but I suspect this file is automatically updated.)

    Thanks for your work!

  • Jon

    Hi Jacob – amazing tool. Thanks for your work.

    I’ve tried a bunch of different installs, but no matter what I do (let’s go with “Setting It Up (The Easy Way)” as that’s what I currently used with raspbian jessie full), the admin interface always shows 0 for everything except the # domains being blocked.

    I tried re-running “sudo touch /var/log/pihole.log sudo chmod 644 /var/log/pihole.log sudo chown dnsmasq:root /var/log/pihole.log” just to make sure, no difference.

    I’m literally using default raspbian and just the simple curl command install. No other changes. Any thoughts? Apart from that, everything seems to be running fine (i.e. by looking at tail of log file).

    Also, I’m very new to linux, but if I wanted to run pi-hole on a regular (not single card pc) machine running ubuntu, how would I go about that? Would the simple curl command work, or follow the manual steps with some modifications?

    • Jon

      Ok, so the log is awash with DNS requests, but blacklisted domains aren’t actually being blocked… So confused, as I have previously had it working from default…

      • jacobsalmela

        Some people have been reporting the admin page showing zeroes except for the domains. The only thing I can think is that the permissions didn’t get applied.

        If you tail -f /var/log/pihole.log, are you seeing the queries come through?

        • Jon

          tail -f /var/log/pihole.log is blank, nothing happens.

          tail -f /var/log/daemon.log shows all the DNS queries though.

          • jacobsalmela

            I have been sending people here. Please let me know if that does not fix the issue.

          • Jon

            Ok, i’m pretty sure the problem is that despite dnsmasq.conf containing “log-facility=/var/log/pihole.log” dnsmasq is logging DNS queries to daemon.log instead of pihole.log. So the console shows 0 because pihole.log is empty. Why would it be logging to daemon when log-facility is set?

          • jacobsalmela

            Well, a lot of changes happened in the script in the last couple of days, but nothing to do with the config. Maybe it just didn’t download everything correctly. I’m glad it’s working now!

    • Jon

      Ok, so the log is awash with DNS requests, but blacklisted domains aren’t actually being blocked… So confused, as I have previously had it working from default…

  • C F

    anyone know of a similar hack for say a dd-wrt capable device?

  • Michiel

    Hi, as I use a similar setup with my banana pi I have experienced that more and more websites are serving now ads over HTTPS (using HTTPS everywhere is a good thing) but this means that solution of blocking ads and malware over HTTP won’t work or won’t be as effective anymore as machines have more CPU power (to handle loads of SSL traffic if needed), bandwidth becomes faster and everyone can get a valid SSL cert nowadays (for not so much money or even for free: https://letsencrypt.org/)
    I do see that dnsmasq will intercept all the requests to the ads/malware lists but you’re connections will fail if you connect to something like googleads over https and need to host this on your own pi…
    I am very interested if this solution will stand its ground in the coming years to come for mentioned reasons (cheap ssl hardware/certs, https everywhere etc.).

    • jacobsalmela

      Yeah, we have been seeing a lot of this lately. We have a few issues on Github open about it. We are looking into some way to block ads over https.

      • Michiel

        To be honest: I think you can’t…
        How will you overcome the SSL cert validation errors? Install your own CA and install the CAs’ cert on every device in your network?
        I am very interested in how you want to solve this since I am also struggling with this question.

  • William Jones

    Love the project. I’m working on installing it on another Pi and get this error:

    bash: line 1: syntax error near unexpected token newline'

    100 1bash: line 1:

    Any thoughts?

    • A Anon

      had the same problem, if you create a new file and paste the github code into the file and then save and set as exe.

      if you right click and save the chronometer.sh file you get bad code inside,.

      • jacobsalmela

        If you want to copy/paste any of the Github code, it’s best to use the raw format.

  • Patrick Pommé

    Thanks for this amazing tool Jacob! Pi-Hole is probably the most meaningful software that mi Pi has ever run… Suggestion: it would be great to have a “disable” and/or “disable for X minutes” button on the webinterface (for instance for sites that block their content when they notice that ads are blocked). Changing the routers DNS config might be a bit too much for some family members 🙂

  • Daan

    I can’t get alternative DNS servers to work 🙁

    After standard installation it works perfect:

    C:>nslookup http://www.google.com
    Server: Pi-Hole.IsWorking.OK
    Address: 192.168.192.24

    Non-authoritative answer:
    Name: http://www.google.com
    Addresses: 2a00:1450:4013:c01::68
    74.125.136.99
    74.125.136.106
    74.125.136.105
    74.125.136.147
    74.125.136.103
    74.125.136.104

    But when I add the script using nano /etc/dnsmasq.conf.

    This is my output:

    C:>nslookup http://www.google.com
    DNS request timed out.
    timeout was 2 seconds.
    Server: UnKnown
    Address: 192.168.192.24

    DNS request timed out.
    timeout was 2 seconds.
    DNS request timed out.
    timeout was 2 seconds.
    DNS request timed out.
    timeout was 2 seconds.
    DNS request timed out.
    timeout was 2 seconds.
    *** Request to UnKnown timed-out

    When I remove the script for alternative DNS, it works again. I’m doing something wrong there, but I have no idea what. The only weird thing I see is that you specify the settings should be changed, I couldn’t find those and I just added those…

  • Daan

    Edit:

    I’ve edited the DNS servers in etcdnsmasq.d1-pihole.conf

    Looks like it works flawless now 🙂

    • jacobsalmela

      Glad to hear it!

  • Andrew

    Hello So I am trying to figure out how to allow video from streaming sites or like the actual networks websites to go trough. Say on NBC.com I am trying to watch an episode of a show and nothing happens when I click on the episode it goes to the page but won’t play anything, Same with Hulu and then on certain cable sites such as USANetwork that require you to login with your provider to watch an episode of a show that doesn’t work because the popup that’s supposed to appear where you can login doesn’t popup. I am assuming it’s being blocked somehow and I am trying to figure out how to let that stuff through. I have tried Whitelising the site so I just added http://www.usanetwork.com/ but it didn’t help so just not sure how to allow that kind of thing to get past the pihole. One thing I notice on the USAnetwork site is when I am on the video and it says click sign in and watch in the bottom left corner it says javascriptvoid(0) so not sure what that could mean.

    Thanks,

    • Simon Lahaye

      Hello Andrew. I have a similar experience with the CBC network and its streaming platfrom “tou.tv” . For exemple, watching a show on the Samsung smart hub app yields the best result. The show streams and ads never manage to load and get skipped. On Firefox through the website, it works too but some ads manage to get through. Finaly, the iOS app is unable to play anything, no ads but no show. So, I don’t know why that is, but keep in mind your milage might vary and you have to try different ways.

    • jacobsalmela

      This has indeed been a challenge (and something I have been working on for a while). I have more information about how to block these but haven’t been completely successful.

      You can try to take use the DNSThingy Whitelist extension to find the domains you may need to whitelist. Since Pi-hole works on the DNS level, it functions differently than most ad-blockers.

  • Steve Wong

    I’m loving the Pi-hole!

    Is there a way to run a script to update to the latest version of the whole setup? Is there a way to tell the current installed version and what is the latest available version? This would be a great thing to show on the web interface (at least the current installed version).

    I’m hoping I can keep my current configuration (I added some custom DNS lookups so don’t want to alter that).

    Thanks again!

    • jacobsalmela

      (sorry for the delayed response) Yes, the versions of Pi-hole and the Web interface are shown on the bottom of the dashboard now.

      You can simply run the installer again until we get our update mechanism working.

  • Tom Hall

    In my out-of-the-box Pi-Hole’s web console, I’m seeing much fewer than the “advertised” 1.6 Million Ad-serving Domains – 146,745 infact.
    I did a nano /etc/pihole/adlists.list, and added the Mahakala list (http://adblock.mahakala.is), and ran Gravity.sh, and that bought the total domains blocked up to 735,511 but that’s still way short
    – any ideas /comments ?

    • jacobsalmela

      Unfortunately, the Mahakala list is the culprit. They change very quickly and without warning. They also block legitimate domains like apple.com and microsoft.com. Some days the list would be around 100,000, while other days it would jump to 900,000. It’s a volitle list and we don’t recommend it. Using the default lists, you can get around 150,000, which still blocks a signifigant amount of ads.

      • Tom Hall

        Thanks I’ll remove it

  • #!/Hugo/Cardozo

    I’ve been using Pi-hole on my BananaPi gateway for over two months now, and it works great.

    Just one caveat: At first some web pages took forever to download (some have ads, some doesn’t). When I learned that pi-hole works mantaining a list of “ad servers” I assumed that was the problem: a potentially HUGE file (my copy has over 80.000 lines now), saved on microSD (a standard installation) and dnsmasq has to look on it for adservers.

    Then I bought a SATA hard drive, moved the OS to it and now surfing the web is blazingly fast 🙂

    Of course the occasional ad comes up once in a while, but what can you do? It’s kind of a “Red Queen” situation: pi-hole blocks more and more adservers, but even more appear every day.

    • jacobsalmela

      Interesting to hear. I have had over a million on my list at one point (on a Pi2). The older B models seem to struggle a little once the lists get that big. Our default list now gives around 150,000, which seems to be a decent amount while providing effecting ad-blocking.

      • Nathan RodHull Adams

        A possible idea to deal with performance issues might be to use a bloom filter, apparently Google Chrome used to use one to check for malicious URLs, would be interesting to see if it helped increase performance for this

        • jacobsalmela

          I understand how they work–just now how to implement one using bash or PHP.

  • maestropaolo

    A simple question. I used the automated install and everything works fine so far, and I’m studying this useful tool. Just for trying, I changed the index.html file in the instructed location to have a background color instead of white…but despite of this I still see white spaces. Am I missing something?

    • jacobsalmela

      Each Website is different and sometimes is hides the space completely, but other times you still see the blank space. You can experiment by changing the index.html file.

  • Sohail Dhanda

    how to install it in dietpi [ w/o using automation ] ? as i tried but it didnt work , after using curl code no webinterface is shown on 192.168.1.*/admin

    • jacobsalmela

      Pi-hole on DietPi is not maintained by the Pi-hole dev team. It is a fork of our project and must be installed using the DietPi installer.

      • Sohail Dhanda

        Well I want to install your version using curl -L install.pi-hole.net | bash . But when I do I am not able to acess Web interface , do I need to install it separately (using diet pi btw ) . I know it’s not maintained by you guys , but as it’s just a strip down version of raspbian I should be able to install it right ?

        • jacobsalmela

          No, the author of DietPi uses Apache and our interface runs off of lighttpd. He doesn’t update his version until he fully tests it with his other software a is going in a different direction then we are. So if you want the new interface, install Raspbian or another Debian-based distro.

  • Vittorio B.

    Hi
    Just installed this nice tool!
    I put on a Raspi B+ that it’s running Domoticz.
    I don’t understand if I have to add sites to be blocked somewhere or, after added the crontab line you mentioned in this article I will have an updated list weekly.
    Another think that I don’t find is the Web Interface. On page 80 I get the Apache page.
    Do II need to set something in the apache config files?

    Thanks
    Vittorio

    • same problem.. automatic update?

      • jacobsalmela

        If you followed the manual setup, it won’t auto update. I stopped updating this post because development was moving too quickly. I suggest installing it via the install command: curl -L https://install.pi-hole.net | bash. This includes cron jobs for updating the lists.

        • Vittorio B.

          I’ve installed via curl -L https://install.pi-hole.net | bash

          but if I go in the cron tab (crontab -e) I don’t find any added line to the ones that I already have.

          • jacobsalmela

            Look at /etc/cron.d/pihole

          • Vittorio B.

            Found it.
            Thanks!!

  • David

    I bought a adtrap and it works great for blocked my ads on my devices but it screws up my cable box commutation w Verizon so my box starts loosing channels then shuts down.. Also my roku player is much slower and refreshing happens often.. Does anyone have a thought on how I can resolve these issues ?

    Dave

    • Simon Lahaye

      Hello David. Adtrap and Pi-Hole are two different things. I have never used the Adtrap, but the Pi-Hole works great for me and my family. If I were you, I would try a Raspbery Pi computer running Pi-Hole and see if it works better than the Adtrap.

    • jacobsalmela

      AdTrap is a slow proxy and has many issues (just read some of the forums). The Pi-hole is a fast DNS server, which only handles DNS traffic, so it won’t slow down your network (it often makes it faster since ads are not being downloaded).

  • Daddyfix

    Just install Pi-Hole on C.H.I.P. (http://getchip.com/). Works Great, Thanks!

  • Les Girauds

    I can’t get to the forum to ask a question – I get ‘Your IP has been flagged for potential security violations’ – so I’ll ask here…

    …has anyone noticed that Chromium displays (some) ads? I can use a different browser on the same URL and no ads will display.

    • jacobsalmela

      What is the URL? We can test it out. Feel free to post an issue on our Github page as well.

      • Les Girauds

        Always happens doesn’t it? The minute you post a question, things start behaving themselves, or misbehaving elsewhere! I’m going to monitor it/check logs because I don’t think it’s as simple as a browser issue – it only seems to be a particular Linux desktop that is having issues. All I can say is that ads are sometimes blocked, sometimes getting through. It’s as if it’s sometimes finding another DNS server to use, despite having static addresses configured.

        • jacobsalmela

          We’d rather know about problems sooner than later! We’re always here to help anyway.

          Is the Pi-hole set as your only DNS server? If not, it may be falling over to a regular server and not blocking ads. Upstream servers are configured in Pi-hole, so you just need to set it as your only server.

          Another thing to try is to flush your DNS cache.

          I know people have had issues with Arch Linux if that’s what you are using.

          • Les Girauds

            The DNS server cannot be changed on my (IP provided) router so I have to set the DNS on each device. That works fine it seems, except for this one desktop. However, I’d just updated pihole to 2.6.1 so maybe a DNS flush was needed too. I’ve done that now so I’ll see how I get on.

  • wb

    after seeing your app and seeing how my AdTrap had a weird 5555 port open(not by me) and AdTrap seems out of business, I ordered a Pi 2 and PiTFT Capacitance screen from adafruit.

    I see you use the Resistive Touchscreen and Adafruit has the custom image build for that.

    will the Capacitance screen work with your app?

    my adtrap is inbetween my cable modem and firewall box. we can put the Pi+PiHole in same spot, yes?
    btw, so glad a better alternative is avail versus the adtrap, have always had some issues with it.

    • jacobsalmela

      It should work just fine, but you may need to tweak some of your screens settings. The script works over SSH without issue, so it’s really just a matter of adjusting the display.

      Also, down with AdTrap!

    • Jan Bigum Warming

      If you just want the easy way you can use https://NoAd.Zone it takes less than a minute

  • frankiecunes

    Great ideas . Speaking of which , people have been needing a 2014 colorado form 105 , my business discovered a fillable form here http://goo.gl/HxiULk.

  • #!/H/a/C

    For some reason the Youtube official Android app stopped working some weeks ago. I try to reproduce videos, it works fine one or two times and then it only says “Error connection, try again”. But if you use a web browser it works fine.

    Did some debugging and saw that before the error the Youtube app tried to access some blacklisted servers… I suppose that if I whitelist all google adservers the app will work again, but as far as I’m concerned, they can go to heck 🙂

  • #!/H/a/C

    I can confirm that pi-hole works great on Armbian (www.armbian.com). Been using a Banana Pi and later an Odroid-C2, both running Armbian.