Failing to Protect a Website From Scraping Not Consent

The Northern District of Illinois recently held that a website operator’s failure to take steps that would have prevented a software program to scrape data from her website did not constitute consent to scrape the data. The plaintiff created a software program that scoured the Internet for information on dog breeds, which it copied into a software product. That product was then sold to the public. Plaintiff’s product accessed defendant’s website and took dog breed information that defendant had compiled. The defendant claimed that the plaintiff stole her information on various message boards and through other communications, and that he engaged in hacking and copyright infringement. The plaintiff subsequently sued the defendant under various theories, including defamation. The plaintiff claimed that its software product did not access websites which used the standard robots.txt protocol (this protocol tells web crawlers no to access a certain page or website), but instead that the defendant’s website did not use the protocol. The court ruled for the defendant on the defamation claim, in part, on the grounds that truth is a defense to defamation. The court determined that nothing indicates “that failure to use the Robot Exclusion Standard means that a website creator has given up her intellectual property rights or invited the general public to copy the material on a site.” Accordingly, the court found the statements by the defendant to be “substantially true.” 

Tip: While the court did not rule specifically that scraping data from another’s website constitutes theft or copyright infringement, as these were not the claims at issue, this case is a reminder that scraping a third party’s website can pose risks, and the mere absence of protection software cannot be presumed – at least for this court – to be consent to scraping.