Wikipedia deprecates Archive.today, starts removing archive links

If they're under an organised defamation campaign, they're not helping themselves by DDoSing someone else's blog and editing archived pages.

archive.today works surprisingly well for me, often succeeding where archive.org fails.

archive.org also complies with takedown requests, so it's worth asking: could the organised campaign against archive.today have something to do with it preserving content that someone wants removed?

There was also the recent news about sites beginning to block the Internet Archive. Feels like we are gearing up for the next phase of the information war.

Was that written by AI? It sounds like AI, spends lots of time summarizing other posts, and has no listed author. My AI alarm is going off.

There are number of blog posts like

owner-archive-today . blogspot . com

2 years old, like J.P's first post on AT

They are able to scrape paywalled sites at random, so im guessing a residential botnet is used.

For a very brief time, "doxing" (that is, dropping dox, that is, dropping docs, or documents) used to mean something useful. You gathered information that was not out in public, for example by talking to people or by stealing it, and put it out in the open.

It's very silly to talk about doxing when all someone has done is gather information anyone else can equally easily obtain, just given enough patience and time, especially when it's information the person in question put out there themselves. If it doesn't take any special skills or connections to obtain the information, but only the inclination to actually perform the research on publicly available data, I don't see what has been done that is unethical.

It's also kind of ironic that a site whose whole premise is to preserve pages forever, whether the people involved like it or not, is seeking to take down another site because they are involved and don't like it. Live by the sword, etc.

Sites that exist to archive other websites will almost always need to dynamically change the content of the HTML that they're serving in some way or another. (For example, a link that points to the root of the website may need changed in order to point to the right location.)

So it doesn't necessarily raise questions about whether the content has been changed or not. The difference is in whether that change is there to make the archive usable - and of course, for archive.today, that's not the case.

> Changing the content of archived pages also raises questions about the authenticity of what we're reading.

This is absolutely the buried lede of this whole saga, and needs to be the focus of conversation in the coming age.

Did they actually run the DDoS via a script or was this a case of inserting a link and many users clicked it? They are substantially different IMO

As far as I understand the person behind archive.today might face jail time if they are found out. You shouldn't be surprised that people lash out when you threaten their life.

I don't think the DDOSing is a very good method for fighting back but I can't blame anyone for trying to survive. They are definitely the victim here.

If that blog really doxxed them out of idle curiosity they are an absolute piece of shit. Though I think this is more of a targeted campaign.

> When I noticed it, it seemed like such a minor change, but with these latest revelations, it doesn't seem so minor anymore.

That doesn't seem nefarious, though. It makes sense they wouldn't want to reveal whatever accounts they use to bypass blocks, and the logged-in account isn't really meaningful content to an archive consumer.

Now, if they were changing the content of a reddit post or comment, that would be an entirely different matter.

It costs money beyond 10 links, which means either a paid subscription or institutional affiliation. This is problematic for an encyclopedia anyone can edit, like Wikipedia.

Does Wikipedia really need to outsource this? They already do basically everything else in-house, even running their own CDN on bare metal, I'm sure they could spin up an archiver which could be implicitly trusted. Bypassing paywalls would be playing with fire though.

[dead]

I like Readeck – https://codeberg.org/readeck/readeck

Open source. Self hosted or managed. Native iOS and Android apps.

Its Content Scripts feature allows custom JS scripts that transform saved content, which could be used to do URL rewriting.

"archive.today" as used here means the collection of archive.tld domains, where .tld could be ".is", ".md", ".ph", etc.

"promoted" as used here means placing an archive.tld URL at the top of an HN thread so that many HN readers will follow it, or placing these URLs elsewhere in threads

I use archive.today all the time. How do you access pages, like for instance on the economist, without it?

you can change the tld of any archive.today link if .today doesn't work. for example archive.ph, archive.is, archive.md, etc

The fact is i cant have a discussion about a paywalled article without reading it. Archive.today is popular as a paywall bypass because nobody wants HN to devolve into debate based on a headline where nobody has rtfa.

> Whomever it is, they have lot of info about HN users' reading habits given that archive.today URLs are so heavily promoted by HN submitters, commenters and moderators

It's not promoted, it's just used as a paywall bypass so everyone can read the linked article.

I'm wondering the same thing, would be great to have something similar for personal use

P2p is generally bad for this usecase. P2P generally only works for keeping popular content around (content gets dropped when the last peer that cares disconnects). If the content was popular it wouldnt need to be archived in the first place.

> I figured that they have found an (automated) way to imitate Googlebot really well.

If a site (or the WAF in front of it) knows what it's doing then you'll never be able to pass as Googlebot, period, because the canonical verification method is a DNS lookup dance which can only succeed if the request came from one of Googlebots dedicated IP addresses. Bingbot is the same.

I imagine accounts are the only way that archive.today works on sites like 404media.co that seem to have server sided paywalls. Similarly, twitter has a completely server sided paywall.

> I've seen people claiming that they have a bunch of paid accounts that they use to fetch the pages, which is, of course, ridiculous.

The curious part is that they allow web scraping arbitrary pages on demand. So if a publisher could put in a lot of arbitrary requests to archive their own pages and see them all coming from a single account or small subset of accounts.

I hope they haven't been stealing cookies from actual users through a botnet or something.

> which is, of course, ridiculous.

Why? in the world of web scrapping this is pretty common.

I’m an outsider with experience building crawlers. You can get pretty far with residential proxies and browser fingerprint optimization. Most of the b-tier publishers use RBC and heuristics that can be “worked around” with moderate effort.

It’s not reliable, in the sense that there are many paywalled sites that it’s unable to archive.

The word salad with ukraine, arms trade, nazis, hunter biden, leave no doubt the operator is from Russia.

He’s probably being purposefully vague which makes for difficult reading.

There was some beef the site owner had with Cloudflare where if your were using Cloudflare DNS it wouldn’t serve anything to you? Is that still happening?

Not sure why it would only be on archive.is and not the others but ‘is’ loads for me.

I would be suprised if archive.today had something that was not in the wayback machine

>> an analysis of existing links has shown that most of its uses can be replaced.

>Oh? Do tell!

They do. In the very next paragraph in fact:

   The guidance says editors can remove Archive.today links when the original 
   source is still online and has identical content; replace the archive link so 
   it points to a different archive site, like the Internet Archive, 
   Ghostarchive, or Megalodon; or “change the original source to something that 
   doesn’t need an archive (e.g., a source that was printed on paper)

The archive.today operator claims on his blog that this was nothing major: https://lj.rossia.org/users/archive_today/

> Regarding the FBI’s request, my understanding is that they were seeking some form of offline action from us — anything from a witness statement (“Yes, this page was saved at such-and-such a time, and no one has accessed or modified it since”) to operational work involving a specific group of users. These users are not necessarily associates of Epstein; among our users who are particularly wary of the FBI, there are also less frequently mentioned groups, such as environmental activists or right-to-repair advocates.

> Since no one was physically present in the United States at that time, however, the matter did not progress further.

> You already know who turned this request into a full-blown panic about “the FBI accusing the archive and preparing to confiscate everything.”

Not sure who he's talking about there.

I prefer archive.today because the Internet Archive’s Wayback Machine allows retrospective removals of archived pages. If a URL has already been crawled and archived, the site owner can later add that URL to robots.txt and request a re-crawl. Once the crawler detects the updated robots.txt, previously stored snapshots of that page can become inaccessible, even if they were captured before the rule was added.

Unfortunately this happens more often than one would expect.

I found this out when I preserved my very first homepage I made as a child on a free hosting service. I archived it on archive.org, and thought it would stay there forever. Then, in 2017 the free host changed the robots.txt, closed all services, and my treasured memory was forever gone from the internet. ;(

It's a reminder how fragile and tenuous are the connections between our browser/client outlays, our societal perceptions of online norms, and our laws.

We live at a moment where it's trivially easy to frame possession of an unsavory (or even illegal) number on another person's storage media, without that person even realizing (and possibly, with some WebRTC craftiness and social engineering, even get them to pass on the taboo payload to others).

I mean, the admin of archive.today might face jail time if deanonymised, kind of understandable he's nervous. Meanwhile for Patokallio it's just curiosity and clicks

That was private negotiations, btw, not public statements.

In response to J.P's blog already framed AT as project grown from a carding forum + pushed his speculations onto ArsTechnica, whose parent company just destroyed 12ft and is on to a new victim. The story is full of untold conflicts of interests covered with soap opera around DDoS.

I know I'm arguing with a bot that nobody monitors, but it's already in the fucking post.

> i don't know anything specific about the site or any conflicts involved, yet this smells like a negative PR campaign to me...

What possible value could a comment from someone who has no knowledge of the site or conflict add to this discussion?

Wayback machine won't bypass paywall nor pirate content, not to mention they are under US jurisdiction. You can't have your cake and eat it.

The objection is to this specific archieve service not archiving in general.

Wikipedia shouldn't allow links to sites which intentionally falsify archived pages and use their visitors to perform DDOS attacks.

They generally do. Random example, citation 349 on the page of George Washington: ""A Brief History of GW"[link]. GW Libraries. Archived[link] from the original on September 14, 2019. Retrieved August 19, 2019."

If you read the linked article it is discussed

Why? The decision seems reasonable at first sight.

About how much had you previously donated over the years?

[flagged]

[flagged]