Comment by jsheard
19 days ago
It won't help with the more egregious scrapers, but this list is handy for telling the ones that do respect robots.txt to kindly fuck off:
19 days ago
It won't help with the more egregious scrapers, but this list is handy for telling the ones that do respect robots.txt to kindly fuck off:
No comments yet
Contribute on Hacker News ↗