Comment by wl
15 hours ago
The Wayback Machine has ignored robots.txt for a few years at this point. The only way to get them to stop scraping or remove content is by asking them directly.
15 hours ago
The Wayback Machine has ignored robots.txt for a few years at this point. The only way to get them to stop scraping or remove content is by asking them directly.
No comments yet
Contribute on Hacker News ↗