Comment by Bender

1 day ago

I disabled blocking for a few hours then crawled several of my pages with their site.

Oh, I assumed Archive had an automated* tool in their arsenal to overcome the blocks that would’ve affected them. Didja happen to confirm they were blocked initially?

*I think they do something manually sometimes, like logging into a The Information Pro account and mirroring popular articles. (Unless they can use a visitor’s account somehow, like with some browser extension…) And suspected without evidence maybe The Information did some watermarking to ensure they could quickly ban accounts used like that.

And thank you!

  • Didja happen to confirm they were blocked initially

    They were the reason I started blocking data-centers. I wanted to see if I could block all of their addresses which turned into quite an exercise. Initially I was able to block them by blocking TCP SYN packets with an MSS of 1380 and 1300 but talking about it here was a mistake as their admin is on HN. They fixed that and made their MSS 1460 like a nominal home internet connection.

    The next thing I had to block was the Russian Federation. That seems to be where the controlling nodes are, I think. That made me even more curious who they were so I just kept iterating through the blocking process. I ended up blocking a number of ASN's and all of Russia. They are all in the linked archive file in the article but I don't specifically call out the ones that are archive.is.

    Most on HN think they operate on a shoe-string budget but I have my doubts. I suspect an asset of a technical arm of an intelligence agency but to what purpose and what end I have no clue. They are very clever whoever they are and certainly earned my respect. Either way I think their site is useful, I just wanted to see if I could block it.

    • > Initially I was able to block them by blocking TCP SYN packets with an MSS of 1380 and 1300 but talking about it here was a mistake as their admin is on HN. They fixed that and made their MSS 1460 like a nominal home internet connection.

      For some reason I find this very amusing.