Nailing Referrer-Spammers
The recent referrer-spam renaissance annoys me. Scumbags like the ten million pixels home page, ads on the moon and zillions more of Alex Tew plagiarists, assclowns like the dutch loan & mortgage scam afab.nl, legions of eBook diddlers like articledirector.com and other make-me-rich-while-screwing schemes, affiliate mobsters like googlecashworks or adsensetoyourincome, link peddlars like belinked.info, and gazillions of other low-life artists are running bots spoofing the HTTP referrer to make me click on their fucking scam URLs in my referrer stats.
Because those suckers are usually smart enough to avoid static IPs, it's hard to block them. However, it's possible to shut them down. I've sent a few cease & desist letters, received apologies, and stopped a couple of referrer spammers. Unfortunately, this approach involves work, that is research.
My emails cc:
· the admin/abuse/hostmaster email address of each domain appearing in the referrers
· the abuse address of the hosting company
· the abuse address of the domain registrar
· all email addresses collected from whois searches including the hosting company
· the abuse addresses of the ISPs from where the bots ran
· the email addresses of local authorities and all sorts of spamcops
Q: How to get the hosting service and domain registrar via domain name?
A: Do a whois search at WebHosting.info
Q: How to get the spammer's ISP?
A: Search your server logs for the faked referrer URL and do a whois search at GeekTools for the user's IP address
Q: What is a C&D letter?
A: Example C&D "Konstantin Lysenko and Sergei Goshko, stop your referrer spam bot or I'll shut you down. 100 faked requests per hour to several URLs not linked from your site with your home page as referrer URL is abusive." Example answer "Dear Sebastian, I worked on tool that checks my clients resources for inappropriate content. Possibly it went out of control. I disabled it for now. I'm really sorry it caused you problems. Thank you, Sergei". The reply is laughable, but at least the referrer spam from that assclown was stopped.
Be creative :)
Because those suckers are usually smart enough to avoid static IPs, it's hard to block them. However, it's possible to shut them down. I've sent a few cease & desist letters, received apologies, and stopped a couple of referrer spammers. Unfortunately, this approach involves work, that is research.
My emails cc:
· the admin/abuse/hostmaster email address of each domain appearing in the referrers
· the abuse address of the hosting company
· the abuse address of the domain registrar
· all email addresses collected from whois searches including the hosting company
· the abuse addresses of the ISPs from where the bots ran
· the email addresses of local authorities and all sorts of spamcops
Q: How to get the hosting service and domain registrar via domain name?
A: Do a whois search at WebHosting.info
Q: How to get the spammer's ISP?
A: Search your server logs for the faked referrer URL and do a whois search at GeekTools for the user's IP address
Q: What is a C&D letter?
A: Example C&D "Konstantin Lysenko and Sergei Goshko, stop your referrer spam bot or I'll shut you down. 100 faked requests per hour to several URLs not linked from your site with your home page as referrer URL is abusive." Example answer "Dear Sebastian, I worked on tool that checks my clients resources for inappropriate content. Possibly it went out of control. I disabled it for now. I'm really sorry it caused you problems. Thank you, Sergei". The reply is laughable, but at least the referrer spam from that assclown was stopped.
Be creative :)
 Stumble It! |
 Post it to del.icio.us |
posted by Sebastian @ Wednesday, January 11, 2006 · PERMANENT LINK
-->
Subscribe to my blog
6 Comments:
At Tuesday, January 24, 2006,
IncrediBILL said…
Sebastian, your post begs the question of why does referrer-spam bother you unless you're one of the many letting the search engines index your web stats?
Can you say web stats password protection using .htaccess?
I know you could
At Tuesday, January 24, 2006,
Sebastian said…
Yep, all stats are pw protected. If not I wouldn't care about the engines crawling them, coz referrer stats don't pass reputation.
Actually, publishing my referrer stats with link condom would be a great idea, the engines would discover huge loads of crap and handle it properly in their graveyards. Thanks for this idea:)
Referrer spammer bother me coz I'm a data rat and collect stats from all page views to measure the value of inbound links etc.
At Wednesday, January 25, 2006,
IncrediBILL said…
Letting engines crawl your stats is never a good idea because your competition can also see your "marketing" plan at a glance.
Then they know directly where to go to compete with you head-to-head without trying to figure out where you get most of your traffic.
At Wednesday, January 25, 2006,
Sebastian said…
Yep, I would not use the real stats, but samples (the spammers) w/o figures linked with condom would ensure the engines pick up the points of entry to a few spam networks.
I've even scripts for fully automated spam reports laying around, those would need very little customizing for an integration in such a fun tool.
The logic behind is very simple. All referrers where my link is not found on the referring page are classified spam. A cron job would submit the spam reports and put the whois info and offending pages on an outing page. It could email the ISPs and domain registrars too and whatever.
I'm pretty much pissed coz on one site those assclowns (scrapers and referrer spammer) burn cpl gigs daily. Yesterday one scraper sucked 0.5 gigs within 15 minutes or so. Too sad I've not the time to code it at the moment. I'm just collecting ideas, and plan to code it in my spare time one day. The pain threshold is not yet reached, and I've enough work with paying visitors :)
At Wednesday, January 25, 2006,
IncrediBILL said…
Hate to rain your parade but busy news aggregator sites and blogs may have a link to you on the referring page but it can scroll of by the time you get there and you'll be generating false positives.
I know this as I've seen it literally happen in a few hours on sites referring to my sites.
Oh well.
Use AlexK's PHP script to stop most of that scraping NOW....
http://www.webmasterworld.com/forum88/7288.htm
A couple of threads on that topic with instructions to auto prepend/append to all web pages site wide and all that.
Not quite as sophisticated as my toy but it will stop a lot of the problems your having.
At Thursday, January 26, 2006,
Sebastian said…
This fun script idea is not yet thought out. Regarding blogs etc. the concept needs refinement, like white lists for social bookmarking services etc. and checking links within minutes. Oh well, like most projects that's a complex thingy, so please don't understand my thoughts as a real concept. Perhaps I'll never realize it, dunno.
Thanks, I've looked at Alex' script before. At this time I wasn't sure whether I'm willing to take the risk of banning "good" bots, so I've linked him on my site and moved on. Not that there are many good bots, probably less than a dozen bots do generate traffic, so an IP white list should do the trick. Sad.
Thank you for your thoughts, I do appreciate your comments, and I do enjoy your blog :)
Post a Comment
<< Home