Comment by jsheard
4 months ago
It won't help with the more egregious scrapers, but this list is handy for telling the ones that do respect robots.txt to kindly fuck off:
4 months ago
It won't help with the more egregious scrapers, but this list is handy for telling the ones that do respect robots.txt to kindly fuck off:
No comments yet
Contribute on Hacker News ↗