mechtopia

Referrer Spam: what do you do?

Back in the early noughties, web traffic was increasing to the point where you had to create summaries of the traffic for analysis. Enter the web stats package.

One thing I noticed at the time was that many web stats packages published their referrers, and that many of the people had (often unintentionally) made their stats public.

Google had become the biggest web search engine on the planet, and this posed some ideas. What if you searched for web stats pages on google, then created a link to that page. you would instantly register a link back to your own site. In fact, why bother with a link? What if you just faked the referrer? you could push your link to the top of their referral chart very quickly with an automated script, and google would register another link to your site, increasing the pagerank.

No matter how appealing the idea, google was already by then battling against underhand techniques, and although at the time I was sure I was the first person to think of the idea (something which, in hindsight is ridiculous) I figured that any new underhand techniques would be factored in to the indexing algorithm, and would come back to bite you on the rear.

I'd forgotten completely about my underhand scheme until the 8th of this month, when I examined my logs to find that I was the victim of this technique; every 50 minutes a host was requesting the root document from my web server, with a referring url that on inspection, did not contain a link to my site. I considered my options. Do I:

1. Ignore it?
2. Block the IP address on the firewall?
3. Install a comprehensive web threat management layer to filter out such requests?
4. Put an exclusion rule for that referrer in the module that generates the referrers?
5. Write a snotty email to the admin contact for the whois entry on the source IP, and cc in the admin contact for the referrer domain?
6. Create a rewrite rule that directs requests from that specific IP to a range of large downloads, in an effort to swamp their broadband connection, or crash the script?
7. Create a script that spams their domain with a referrer of wwww.stopspammingmysite.com?

What would you do? answers on a postcard please. Next week I'll tell you what I did, and what happened.

Posted at 11:44PM Nov 18, 2009 by ohp in General  |  Comments[0]