Legal Commentary

Sheep in Spiders? Clothing?

By Stewart Baker

The malware arms race continues apace.  First, the bad guys hid malware on innocent websites, hoping to infect casual web users.  Next, Google and Yahoo rode to the rescue, using their spiders to find infected websites and warn web users away from the malware.

Now the bad guys have made their countermove. They?re deploying software on infected sites that watches for the search giants? spiders.  When they spot a spider, they serve up bland, malware-free content.  The rest of us get the nasty stuff.

You can see what will come next:  The search giants will try to make their spiders look like ordinary web surfers ? varying their IP addresses, configurations, and other identifiers to avoid detection by the new, more sophisticated malware.

That?s good, but I?ve got a question.  Why can?t we take this strategy one step further?  Why can?t Google and Yahoo release software that lets the rest of us look more like search engine spiders?

It seems to me that that would have two good results.  First, if my browser looks like an antimalware spider, infected sites won?t serve me malware.  Second, Google and Yahoo will find it a lot easier to make their spiders look like the rest of us if the rest of us look like their spiders.

In fact, take it even further.  I?d happily run a bit of code from the search engines that actually watched for malware and reported it to Google and Yahoo when I encountered infected sites.  The bad guys will have a lot more trouble distinguishing between antimalware spiders and ordinary users once the ordinary users actually become spiders.

(Crossposted to www.skatingonstilts.com) (Updated to fix broken link; thanks, Evan)

Full post as published by The Volokh Conspiracy on April 24, 2010 (boomark / email).