Battle of the “bots”
Cats: odd things|Every blogger knows the thrill of checking their stats first thing in the morning and seeing more hits than usual. (GT averages 150-200 hits per day with spikes on Fridays.) But what happens when you see hundreds of hits more than usual?
This very thing occurred this morning. What was going on? Well, if ever there was a time for one of the new words in the 21st century lexicon, this was it. GT was “spidered.”
Spiders are search bots that review blog content. The good ones are welcome because they alert search engines like Google. But the bad ones get up to no good, and go up one side of your blog and down the other for purposes that don’t always work in your favour.
Not only does it warp your stats, it also chews up precious bandwidth (which recently landed GT in trouble. This, BTW, accounts for the dramatic drop in images on this blog – I’m working on a Flickr solution to the problem.)
The spider in question is called Linkwalker 2.0. A quick Google search indicates that is indeed a naughty bot that doesn’t follow the proper search protocols. It is used to search blogs for brand names. The information is then sold to clients who are seeking this type of information.
Not surprisingly, GT hasn’t yet received a thank you letter from the companies that are helping themselves to my bandwidth to enhance their sales intelligence. In the meantime I’ve banned the IP address (209.167.50.27) and I’m posting this to warn everyone else.
Insights on this vexing problem are welcome.