Comment by arjie
7 hours ago
I do like the Internet Archive but I wish I could retrieve large segments from it myself. It has very aggressive 429s. It would be a nice higher donor tier feature. Maybe it’s not possible to do so and retain 501c3 status? Regardless I have had to start performing personal datahoarding. C’est la vie.
They posted on X about supporting this when someone was spamming them with requests
I was looking for some CNATRA (US naval aviation) unclassified, non-distribution-controlled training manuals and noticed that the newest ones had been deindexed when they were put behind authentication.
Stop hiding public data and I might reconsider.
I wonder if they could sell you access to a bittorrent tracker? Officially it would be independent of the donations, just an extra service they sell?
They have torrent downloads for a lot of content, I've downloaded almost a terabyte from them via torrent, they're just not bundled, as in you would need to retrieve each torrent file for each item individually, which might still get you a 429 if you automated it idk.
I've never actually got a 429 using a browser though personally, they do throttle aggressively though