Comment by jsheard
1 year ago
It won't help with the more egregious scrapers, but this list is handy for telling the ones that do respect robots.txt to kindly fuck off:
1 year ago
It won't help with the more egregious scrapers, but this list is handy for telling the ones that do respect robots.txt to kindly fuck off:
No comments yet
Contribute on Hacker News ↗