Comment by StrauXX
2 years ago
IANAL but as far as I understand the current legal status (in the US) a change in robots.txt or terms and conditions is not binding for web scrapers since the data is publicly accessible. Neither does displaying a banner "By using this site you accept our terms and conditions" change anything about that. The only thing that can make these kinds of terms binding is if the data is only accessible after proactively accepting terms. For instance by restricting the website until one has created an account. Linkedin lost a case against a startup scraping and indexing their data because of that a few years ago.
At the federal level; but states have their own laws. For instance, it can get you 5 years in prison in Illinois to violate a web site ToS.
https://www.ilga.gov/legislation/ilcs/ilcs4.asp?DocName=0720...
Has anyone ever successfully been prosecuted for violating this statute?
I don't know. That data is really hard to put together.
1 reply →
Quite sure they are also enforcing these with some technical measures to limit scraping.
As was LinkedIn, who was forced to rate stop limiting / IP-banning scrapers for public pages.
Many other websites enforce rate limiting or IP banning for public pages (using products like Cloudflare). Why is this not legal?
Really? That seems strange.