Attempting to Block Hulu Plus Video Ads Using A Raspberry Pi

[UPDATE 2015-03-28] I have some more information about blocking video ads.  I just need to make some time to write it up…

The keyword here is attempting.  What is strange is that I actually had this working at one point.  The Hulu video ads would be completely skipped (not just 30-90 seconds of silence).  Unfortunately, when I reinstalled Raspbian and tried to follow my own steps (a few weeks later), it no longer worked.  Since then, I have gone through the steps many, many times and have come to the conclusion that Hulu must have changed something on their end that disrupted my method, or I forgot to document a crucial step…

The basic concept was to deliver your own streaming content (stored on the Raspberry Pi) in place of Hulu’s video ads.

Disclaimer: It Worked Once, But I Cannot Reproduce The Results

As of now, this article is posted more for informational purposes as opposed to a working solution.  I was able to completely skip the ads using this method, but could not once I followed through my own walkthrough.  Maybe I missed something along the way, but I am hoping someone out there will know more than me and help figure out a way to block them.

Install With One Command (Someday…)

I also liked this idea so much, that I had it set up to be able to be installed with a single command.  If I ever get it working, I will post that command here.

Conceptual Overview

When I had this working, it would use DNS spoofing and then deliver my own streaming content in place of Hulu’s video ads.  The workflow went like this:

  1. If the DNS server running on your Raspberry Pi determines that the query is a URL known to serve ads, it redirects the client to the Pi instead of the real location
  2. A lighttpd  Webserver running on the same machine (the Pi) then redirects that request to another URL, which delivers streaming content via minidlna .

pihole-setup

In theory, everything sounds like it should work.  Earlier, I speculated that the Hulu video ads might not be getting skipped because they needed to have streaming video content instead of a static Webpage.  I still think this is true because I did have it working at one point.  Give the steps below a try and let me know in the comments if you have any success.  I am curious to know if anyone else can get it working.

Requirements For This Walkthrough

This is the culmination of the Pi-hole ad-blocker and is an fairly technical setup.  As I mentioned, I had it working at one point, but when I tried to replicate it, I could not.  I feel like Hulu might have changed something, or maybe a piece of the software used in this tutorial was updated/changed.  Please follow the steps below and keep in mind that it might not actually skip the ads.  Nevertheless, I wanted to post my efforts in case someone smarter knows what I might be missing.

Materials

  1. Local network
  2. Mac (or PC)
  3. Raspberry Pi set up as a pi-hole

Resources

Downloads

  1. VLC
  2. pi-hole.mov

How To Block Hulu Plus Ads On A Raspberry Pi

If you already have set up the Raspberry Pi as an ad-trap, you might just need to modify some of the steps below but this walkthrough can be done as a stand-alone project, so you may want to start from a fresh, clean install of Raspiban.  You will need both dnsmasq  and the lighttpd  Web server for this walkthrough, as well as minidlna .

Conceptual Overview

  1. Set up your Pi as a DNS server
  2. Set up your Pi as a Webserver
  3. Set up MiniDLNA to serve streaming video content to your devices
  4. Upload a very short video to the Pi that will be shown instead of the Hulu ads
  5. Re-direct Hulu’s ad URLs to the page hosting the video uploaded in step 2
  6. Set your device to use the Raspberry Pi as its DNS server
  7. (Almost) never see an annoying video ad again

Install dnsmasq

Run these commands to install dnsmasq

sudo apt-get -y install dnsutils dnsmasq
sudo service dnsmasq stop

Configure DNS

Make a backup of the original dnsmasq.conf  file so you have something to revert to if you make a mistake.

sudo mv /etc/dnsmasq.conf /etc/dnsmasq.conf.orig

Then, edit a new config file with the following content (note that this is slightly different than the previous setups):

sudo vi /etc/dnsmasq.conf
no-resolv
no-poll
# Change the IP below to the IP of your Raspberry Pi
server=/localnet/192.168.0.1
server=/localnet/8.8.8.8
server=/localnet/8.8.4.4
domain-needed
interface=eth0
strict-order
min-port=4096
cache-size=10000
log-queries
bogus-priv

To see what some these options mean, check out the original Pi-hole ad blocker.

Configure the Hulu Ad URLs

Next, create a list of known ad URLs that will be redirected when queried by DNS.

sudo vi /etc/dnsmasq.d/huluAdList.conf

You can add two different things to this file.  If you strictly want to block Hulu video ads and nothing else, make a file with just the Hulu ad URLs, which I will discuss in a moment.  Or you can run the gravity-adv.sh script, which pulls ad URLs from many different sources (including many Hulu ones).  Note that this script will create a file called adList.conf  instead of huluAdList.conf .  Both files should be able to co-exist, but I don’t know what happens if there are duplicate entries in each.

So according to Hulu’s information, in order for ads to get through, the following two domains need to be unblocked:

  • p.hulu.com
  • ads.hulu.com

I set up my Pi-hole to block just those two domains, and I can see that it is actually serving up my streaming video instead of the ad image as you can see in the screenshot below:

pi-hole blocks hulu

So I know that the Pi-hole is working, but when you actually try to do this while streaming a video, the effect doesn’t quite work.  On a few occasions, I saw the Pi-hole image for a moment and then it would disappear.  I know I am close to a solution, but just can’t figure out that last piece of the puzzle.

If you actually watch the log file when streaming a Hulu video, you will see that there are a lot more URLs being queried then just the two Hulu tells you about.  My guess is as good as anyone’s on which ones need to be blocked.  Here is a list of ones that I think are ad URLs.

a.dlqm.net
ads-a-darwin.hulu.com
ads-bid.l.doubleclick.net
ads-v-darwin.hulu.com
ads-v-darwin.hulu.com.c.footprint.net
amch.questionmarket.com
assets.huluim.com
bs.serving-sys.com
cdn.innovid.com
core.insightexpressai.com
dart.l.doubleclick.net
data.agkn.com
ds.serving-sys.com
dts.innovid.com
googleads4.g.doubleclick.net
hulu-darwin.fms.c.footprint.net
hulu-pubnet.telemetryaudit.com
p.hulu.com
pagead.l.doubleclick.net
pagead.l.doubleclick.net
s.innovid.com
sb.scorecardresearch.com
secure-us.imrworldwide.com
stats.l.doubleclick.net
t.hulu.com
t2.hulu.com
tapestry.tapad.com
telemetry.huluad.com
ts.vindicosuite.com
view.atdmt.com.nsatc.net
vindicoasset.edgesuite.net

The /etc/dnsmasq.d/huluAdList.conf  will need these domains to be in the format:

address=/domain.com/192.168.1.101

Just change the IP address to the address of your Raspberry Pi.  See an example of this file here.

Install MiniDLNA

Install MiniDLNA

sudo apt-get install minidlna

Create A Folder To Store The Pi-hole Video

sudo mkdir -p /var/lib/minidlna/videos

Edit the MiniDLNA Config File

Make a backup of the config file in case you mess up:

sudo cp /etc/minidlna.conf /etc/minidlna.conf.orig

and then run this command to see what is currently enabled in the config file (it might be nothing):

cat /etc/minidlna.conf | grep -v "#" | sed '/^$/d'

Edit the file (uncommenting items, if necessary) similar to the following.  You can run the command from above to verify everything matches.  Alternatively, you can just create a brand new file with the information below instead of finding and uncommenting features.

sudo vi /etc/minidlna.conf
media_dir=V,/var/lib/minidlna/videos/
port=8200
friendly_name=pihole
serial=12345678
model_number=1
inotify=yes

Below are what each of the options mean:

  • media_dir=: this is where the video files are stored.  The capital “V” tells minidlna that this folder will only contain video files.  You can also add “A” for audio, or “P” for pictures.  The letter is followed by a comma, and then the path to the folder.
  • port: port to access from a Webpage
  • friendly_name: this will show up in VLC so it is easy to find
  • serial: reported to clients (might not even be necessary)
  • model_number: also reported to clients
  • inotify: watches paths for changes (additions of files)

Start the streaming service

sudo service minidlna start

You can verify the streaming service is working by navigating to the Pi’s IP address at port 8200 :

http://192.168.1.101:8200/

minidlnasererrunning

Copy A Small Video File Over To The Raspberry Pi

There is nothing on the server yet, as seen in the screenshot above, so we need to add a file.  For this project, we only need one.  It will be a very short video that will take the place of the Hulu ads.

Put The Video In A Folder So MiniDLNA Can Find It

The easiest way to get this file is to run a curl  command from the Pi.

sudo curl -o /var/lib/minidlna/videos/pi-hole.mov "https://dl.dropboxusercontent.com/u/16366947/Files/pihole.mp4"

This will just download it directly to your Pi in the correct folder.

If you have the file on another local machine, copy the pi-hole.mov over to the Pi using scp .  To send the file to the pi from another computer use this command.

scp ~/Downloads/pi-hole.mov pi@192.168.1.101:/var/lib/minidlna/videos

to get the file (while on the Pi), use this command

scp <YOUR_-- USERNAME>@<IP_ADDRESS>:/Users/<YOUR_-- USER>/Downloads/pi-hole.mov /var/lib/minidlna/videos

Reload MiniDLNA

sudo service minidlna force-reload

Check that it found a file by running this command:

tail -f /var/log/minidlna.log

You should see an entry showing that it found a file (highlighted below)

.......
[2014/10/11 03:59:27] minidlna.c:1006: warn: HTTP listening on port 8200
[2014/10/11 03:59:27] scanner.c:727: warn: Scanning /var/lib/minidlna/videos
[2014/10/11 03:59:28] scanner.c:798: warn: Scanning /var/lib/minidlna/videos finished (1 files)!
[2014/10/11 03:59:28] playlist.c:125: warn: Parsing playlists...
.......

You can also check the Webpage again to see if it found a video file.  If you used a different video than the one provided in the Downloads section (or from the curl command), and it didn’t show up, it might not be formatted correctly or it might be too big.  I originally tried it with an MP4 from Final Cut Pro X, but it didn’t work.  I exported the same movie as the lowest resolution possible and it worked.

Find The URL Of The Video

  1. Open VLC and click Universal Plug ‘n Play (UPnP)
  2. The expand pi-hole > Video > All Video > pi-hole.mov
  3. Right-click it and choose Media Information.

This will show you the URL of the video file.  This URL will be needed in the lighttpd  config file (this will serve requests for Hulu ads with the Pi-hole video instead of the ad).

vlc_find-video

Configure lighttpd To Redirect Hulu Ads To The Pi-hole Video Instead Of Your TV Screen

This is where the magic is made.  Edit the lighttpd  config file.  We are going to tell the Webserver to redirect any Hulu Plus ad URLs to the video we uploaded earlier.

sudo vi /etc/lighttpd/lighttpd.conf

Make sure mod_redirect  is uncommented:

server.modules = (
        "mod_access",
        "mod_alias",
        "mod_compress",
        "mod_redirect",
#       "mod_rewrite",
)

At the end of the config file, you will need to add entries similar to the following.  One for each Hulu ad URL you intend to block with a pipe |  in between them.  Your URL from before goes on the redirect line in between the quotes.

$HTTP["host"] == "ads.hulu.com|p.hulu.com" {
url.redirect = (
"^/(.*)" => "http://192.168.1.101:8200/MediaItems/19.mov",
)

This tells lighttpd  that if someone is asking for the content at ads.hulu.com or p.hulu.com, they should instead get redirected to the movie file running off of MiniDLNA.

The /usr/local/bin/gravity.sh  script that I made will add a bunch of Hulu urls to the list.  You can see some of them by running the file gets created.

cat /etc/dnsmasq.d/adList.conf | grep hulu

This will return a list of “known” Hulu ad URLs that are already set to be routed to the Raspberry Pi instead of the real location.

address=/ads.hulu.com/192.168.1.101
address=/ads-v-darwin.hulu.com/192.168.1.101
address=/a.huluad.com/192.168.1.101
address=/ads.hulu.com/192.168.1.101
address=/ads-v-darwin.hulu.com/192.168.1.101
address=/a.huluad.com/192.168.1.101
address=/hulu.112.2o7.net/192.168.1.101
address=/huludev.112.2o7.net/192.168.1.101
address=/ll.a.hulu.com/192.168.1.101
address=/t2.hulu.com/192.168.1.101
address=/t2.huluim.com/192.168.1.101
address=/t-ak.hulu.com/192.168.1.101
address=/track.hulu.com/192.168.1.101
address=/tw.i.hulu.com/192.168.1.101
address=/urlcheck.hulu.com/192.168.1.101

 Some Hulu Video Ads Are Still Getting Through

The list above is not a be-all end-all aggregation.  The ads also get served from other third-party sites that may not appear on the list.  If you watch the dnsmasq  log file while an ad is playing on Hulu, you will see all the URLs being queried.

tail -f /var/log/daemon.log

From there, you can start appending these URLs to the /etc/dnsmasq.d/adList.conf  list.  If you don’t add them here, DNS won’t know to route them to the Raspberry Pi instead of the real location.  But, you also need to add the domain to the /etc/lighttpd/lighttpd.conf  file.  Both parts are necessary for the video ads to be neutralized.

Testing Redirection

You can test if your redirection is working with a Terminal command from a Mac on the same network (make sure the Pi is set as this computers DNS server).

curl -I ads.hulu.com

If it is set up correctly, you will see output like this:

HTTP/1.1 301 Moved Permanently
Location: http://192.168.1.101:8200/MediaItems/19.mov
Date: Wed, 19 Nov 2014 00:11:52 GMT

If it is not working, you will see

HTTP/1.1 403 Forbidden
Server: nginx/1.4.2
Content-Type: text/html
Content-Length: 168
Cache-Control: max-age=31536000
Date: Tue, 18 Nov 2014 15:13:38 GMT
Connection: keep-alive

If you were able to skip the Hulu ads, please let me know in the comments what you did differently.

17 Replies to “Attempting to Block Hulu Plus Video Ads Using A Raspberry Pi”

    1. Please do. I was working on it for months, but couldn’t figure out what step I was missing. I had to step away for a while because my brain was hurting. Please let me know what you find out.

    1. And it skips the ads or plays the Pi-hole movie instead? You never end up with the black screen saying you need to enable ads?

      It looks like your file will redirect Hulu domains and then direct other ads to the Pi-hole. Is that correct?

      This is awesome if you got it working. Ever since Hulu has been increasing their ads from 1 of 2 to 1 of 5, I really want to get this working.

      On a side note, do you know if there is a way to store the redirect rules in a separate file, similar to how dnsmasq can have its config files in /etc/dnsmas.d/? I was just thinking of this in case Hulu changes their domains and also for readability.

      1. fyi, i’m running a virtualbox image of ubuntu, but the steps have worked.

        on the xbox one, it plays the movie from the lighttpd server (fyi, i made a .25 second movie out of a black jpg, but yours works). because i have redirected all hulu ads to that mov file, it would not be good to use from the pc. that list would have to be trimmed down otherwise EVERY ad from hulu would show that video (including banner ads).

        I’m not sure about storing the redirect rules in another file, i did however update the host filtering to any that include hulu as seen here: http://pastebin.com/j1nPVEcg

        I get my raspberry in the mail tomorrow, and am planning on setting it up as a wifi access point that i can point my xbox and roku to so that only they get the ads blocked

        1. Interesting. I’ll have to try this out this weekend and see if I can duplicate your results. If so, I will write up a full walkthrough.

          1. I’m eager to try this. I was able to get ad blocking to work using Ubuntu on Parallels, but I couldn’t prevent the adblock warning on Hulu nor block video ads. Might have been a problem with my config though.

          2. Are you using the minidlna setup I describe in this article or some other method?

          3. Cool. I’m hoping to revisit this soon as the Pi-hole has had some improvements recently. Let me know if you get it to go.

  1. on the xbox one, it plays the movie from the lighttpd server (fyi, i made a .25 second movie out of a black jpg, but yours works). because i have redirected all hulu ads to that mov file, it would not be good to use from the pc. that list would have to be trimmed down otherwise EVERY ad from hulu would show that video (including banner ads)

  2. Hi everyone,

    I’m also struggling with Pi-Hole and Hulu ads, and I’ve found same issues as Jacob: my platform works well in Hulu Help page (as displayed in this page), but no Hulu app plays nice with it. I’ve followed Mr. Salmela work to the point, but I’ve also tried Mr. Gentry’s files.

    I’ve tried so far: PS3, XboxOne, iOS, ATV3, web browser, etc.

    So, my thoughts:

    – You’re using .mov files, but I couldn’t download pi-hole.mov. Does this file need any special property? i.e. bitrate, resolution, etc. Mr. Gentry, how did you create your own .mov file? Do different platforms need different type of files? Like .mov with iOS or .wmv with Microsoft..

    – Another for Mr. Gentry: does your implementation still works? Are you still using Virtual Machine? Raspberry?

    – Could it be platform related? With pixelsrv it worked for a while with Xbox360, but zero luck with PS3 or iOS..I don’t have any Android device at hand, but I’m planning make some test with them..

    – Mr. Salmela, Have you made any other improvement, as per [UPDATE 2015-03-28]?

    And most important, thank you very much for your time and sharing your knowledge 🙂

    1. I have had mixed results using a few different methods. Most recently, I have been testing this method, but it is slow going. Sometimes I wish I could make a living doing stuff like this, but alas, it will just have to fit into life as it can!

      1. Thanks! I’ll try it ASAP.

        Long time ago I also had mixed result, as client in Xbox360 being blocked and web player failing. So maybe Xbox client works differently..

        Yeah, I’ll gladly pay for a product like that, specially for Hulu…Are donations allowed? I can’t find your Paypal button 🙂

        1. I’ll add a PayPal button at some point on pi-hole.net, which will hopefully be the landing page. For now, I’m just doing it for fun until there are enough people interested in it.

          1. Hi,
            I’ve just tried your recommendations, but I hit a wall serving MP4 directly from lighttpd. If I try http://X.X.X.X/black.mp4 from a browser, it works, but from Hulu Help Center web (as showed in your screenshot) I got 404..Weird..

            I also replaced .mov in minidlna with .mp4, and tried Xbox,PS3,ATV3,iOS,Web, and none worked (error 5003). But I tried an old version of Hulu app (2.17) in Android, and it worked beautifully! Ads are completely gone, with 1 sec of black screen 🙂

            I’ll keep working on it, as Android Hulu app is almost unusable from Android set-top-box. If I get it working with any other client (specially Xbox), I’ll be satisfied.

            If I can contribute in any way, please let my now. And thanks again 🙂

          2. Yeah, I have found .mp4 works better most of the time. But it is interesting that the older version of the app works.

            Whatever you do figure out, please post it here or make a pull request on my Github page.

Leave a Reply