Messing With Web Filtering Gateways

Wed, 14 Jan 2009 17:49:26 GMT
by pagvac

Most of us are familiar with several techniques that allow us to bypass web filtering gateways like CS MIMESweeper.

The following are some of them:

  1. access the desired site via IP address rather than domain name
  2. access cached content rather than live data. i.e.: using Google's [cache:]( command
  3. using proxies. i.e.: anonymouse, Google translator, etc
  4. using alternative connections. i.e.: connecting your laptop online via your mobile/cell phone's HSDPA interface

Each method has different advantages and disadvantages. For instance, method #1 only works on servers that do NOT use domain-based virtual hosts, i.e.: shared hosting. The exception to this rule is that the site served by default when requesting the IP-based URL (rather than domain-based), is the one you're after. You'll have to use your judgment when deciding which technique is the right one for you.

Whatever the reason may be, there are many legitimate reasons for accessing websites that are blocked by the gateway in question. Personally, when I'm doing on-site pentests, I sometimes need to access useful online resources, which unfortunately are often flagged under the "hacking" category.

Another nifty trick

There is perhaps a lesser known technique which although does not work against all appliances, it does work even in cases where the web server you want to connect to uses domain-based virtual hosts. I've personally seen work on a Clearswift MIMEsweeper environment. Note that it might not work against the latest versions, so please keep this in mind if you can't replicate this technique!

The idea is to sneak the domain name matching the server's virtual host, while being able to bypass the content filter. As you know, filtering gateways block bad websites based on domain names. For instance, an HTTP request would be inspected to make sure that the requested URL doesn't contain a black-listed domain name.

Not too long ago I tested a MIMEsweeper appliance and noticed that HTTP requests were only inspected for bad domain names within the URL data, but not within the Host: header, i.e.:

[some headers removed for clarity purposes]

The previous HTTP request would bypass MIMEsweeper's filter (not sure if it works on all versions) even if was a black-listed domain. Reason for that is because only the http:// URL is being inspected. The remote server would still happily return the website we're interested in as we have successfully established a TCP connection, and the desired virtual host has been requested.

I put the following steps together to test this technique using Firefox's Modify Headers extension. Please see the attached screenshots for more details:

  1. Get the target site's IP address by using a command line tool such as ping or host, or public websites such as
  2. Fire up Modify Headers
  3. Add a new modify rule (top-left drop-down menu) and enter Host as a name, and the domain name of the site you want to visit as value
  4. Double-click on the new rule so that the red light becomes green (rule is now active)
  5. If the technique worked against your appliance, you should now be able to freely browse the blocked site by entering its corresponding IP address in your browser's address bar

And this is one of the many techniques to bypass web filtering gateways.

Archived Comments

I threw together a 5-part series on bypassing web filters, and I missed some of these. There are indeed a myriad of ways. Some others worth mentioning:
  • Web-Enabled RSS (Google Reader, yahoo pipes, etc)
  • Out-of-band (EV-DO, Neighboring WiFi, etc)
  • Web-Enabled Anonymizers (PHP Proxy, Megaproxy)
  • Wide-open public proxies on Port 80 ( for lists of these)
  • Tunneling (SSH, ICMP, SSL/HTTP)
Indeed, it's quite hard to keep people from accessing content that they really, really want to access. Good write-up.
Adrian CrenshawAdrian Crenshaw
Thanks for using me as the test subject. :)
I know tha the topic is ralated to proxy bypass techinichs, but.. a example of Tunneling is that tool.. UltraSurf It´s very usefull when the objectve is to avoid proxy restrictions of urls and you dont have much time to do it yourself.
In my high school the admins block "internet traffic" using dans guardian to filter http requests. I noticed how I could freely use ftp or ssh so I'm using a couple of "header distorcing proxies" to bypass the filter: one application runs on the pc i'm using at school increasing by one the value of each letter of the first word in the headers ( HTTP -> IUUQ ) and then passing the stream to a remote proxy which shifts the header back ( IUUQ -> HTTP ). The reverse is done for incoming responses. What do you think about this? Every feedback is welcomed!!
Franck WurtzFranck Wurtz
A good way to surf all days bypassing the webfitering gateway is to used proxy tunneling. Here is a demonstration :
Although ISP's and admins configure to filter such websites, data sniffing while filtering out and writing the requests to rawdata file and analysing mechanism can trace the users requesting website in order to block them. hence i recommend using a trusted Secured anonymous tunneling like which is encrypted and which cannot be logged for requested websites. any other ideas as i suspect my ISP sniffing out for the websites i access.