Wikipedia deprecates Archive.today, starts removing archive links

If they're under an organised defamation campaign, they're not helping themselves by DDoSing someone else's blog and editing archived pages.

archive.today works surprisingly well for me, often succeeding where archive.org fails.

archive.org also complies with takedown requests, so it's worth asking: could the organised campaign against archive.today have something to do with it preserving content that someone wants removed?

There was also the recent news about sites beginning to block the Internet Archive. Feels like we are gearing up for the next phase of the information war.

Was that written by AI? It sounds like AI, spends lots of time summarizing other posts, and has no listed author. My AI alarm is going off.

A big fear of mine is something happening to archive.is

There is so much is archived there, to lose it all would be a tragedy.

There are number of blog posts like

owner-archive-today . blogspot . com

2 years old, like J.P's first post on AT

They are able to scrape paywalled sites at random, so im guessing a residential botnet is used.

For a very brief time, "doxing" (that is, dropping dox, that is, dropping docs, or documents) used to mean something useful. You gathered information that was not out in public, for example by talking to people or by stealing it, and put it out in the open.

It's very silly to talk about doxing when all someone has done is gather information anyone else can equally easily obtain, just given enough patience and time, especially when it's information the person in question put out there themselves. If it doesn't take any special skills or connections to obtain the information, but only the inclination to actually perform the research on publicly available data, I don't see what has been done that is unethical.

It's also kind of ironic that a site whose whole premise is to preserve pages forever, whether the people involved like it or not, is seeking to take down another site because they are involved and don't like it. Live by the sword, etc.

> Changing the content of archived pages also raises questions about the authenticity of what we're reading.

This is absolutely the buried lede of this whole saga, and needs to be the focus of conversation in the coming age.

Sites that exist to archive other websites will almost always need to dynamically change the content of the HTML that they're serving in some way or another. (For example, a link that points to the root of the website may need changed in order to point to the right location.)

So it doesn't necessarily raise questions about whether the content has been changed or not. The difference is in whether that change is there to make the archive usable - and of course, for archive.today, that's not the case.

Did they actually run the DDoS via a script or was this a case of inserting a link and many users clicked it? They are substantially different IMO

As far as I understand the person behind archive.today might face jail time if they are found out. You shouldn't be surprised that people lash out when you threaten their life.

I don't think the DDOSing is a very good method for fighting back but I can't blame anyone for trying to survive. They are definitely the victim here.

If that blog really doxxed them out of idle curiosity they are an absolute piece of shit. Though I think this is more of a targeted campaign.

this seems like type of thing that should be on blockchain and decentralized nodes validate authenticity, it could support revisions but not lose originals

It could be a donated account. I've noticed archive.whatever also bypasses some paywalls by using legitimate account logins but I doubt there's one person going around subscribing to every news outlet that gets any coverage.

If archive.whatever wasn't so useful to the general public, it'd be hard to distinguish from a criminal operation given the way it operates, unlike say the Internet Archive who goes through all of the proper legal paperwork to be a real nonprofit.

Lead to what?

> When I noticed it, it seemed like such a minor change, but with these latest revelations, it doesn't seem so minor anymore.

That doesn't seem nefarious, though. It makes sense they wouldn't want to reveal whatever accounts they use to bypass blocks, and the logged-in account isn't really meaningful content to an archive consumer.

Now, if they were changing the content of a reddit post or comment, that would be an entirely different matter.

It costs money beyond 10 links, which means either a paid subscription or institutional affiliation. This is problematic for an encyclopedia anyone can edit, like Wikipedia.

I switched to Perma.cc earlier this week and have had a mixed experience to say the least. I think image heavy pages just error out completely, while still charging me such as:

https://www.in.gov/nircc/planning/highway/traffic-data/inter...

and reddit blocks their agent seemingly. It is open source though.

Does Wikipedia really need to outsource this? They already do basically everything else in-house, even running their own CDN on bare metal, I'm sure they could spin up an archiver which could be implicitly trusted. Bypassing paywalls would be playing with fire though.

[dead]

The word salad with ukraine, arms trade, nazis, hunter biden, leave no doubt the operator is from Russia.

He’s probably being purposefully vague which makes for difficult reading.

I like Readeck – https://codeberg.org/readeck/readeck

Open source. Self hosted or managed. Native iOS and Android apps.

Its Content Scripts feature allows custom JS scripts that transform saved content, which could be used to do URL rewriting.

Omnom comes to mind:

* https://omnom.zone/

* https://github.com/asciimoo/omnom

> Change the original source to something that doesn't need an archive (e.g., a source that was printed on paper), or for which a link to an archive is only a matter of convenience.

They're basically recommending changing verifiable references that can easily be cross-checked and verified, to "printed on paper" sources that could likely never be verified by any other Wikipedian, and can easily be used to provide a falsification and bias that could go unnoticed for extended periods of time.

Honestly, that's all you need to know about Wikipedia.

The "altered" allegation is also disingenuous. The reason archive.org never works, is precisely because it doesn't alter the pages enough. There's no evidence that archive.today has altered any actual main content they've archived; altering the hidden fields, usernames and paywalls, as well as random presentation elements to make the page look properly, doesn't really count as "altered" in my book, yet that's precisely what the allegation amounts to.

Archive.today wants/needs EDNS subnet

"Geolocation" as a justication is ambiguous

Why a need for geolocation

Geolocation can be used for multiple purposes

"DNS performance" is only one purpose

Other purposes might offer the user no benefit, and might even be undesirable for users

As a result, some users don't send EDNS subnet. It's always been optional to send it

Even public resolvers, third party DNS services, like Cloudflare, recognise the tradeoffs for users and allow users to avoid sending it. Popular DNS software makes compiling support for EDNS subnet optional

Archive.today wants/needs EDNS subnet so bad it tries to gather it using a tracking pixel or it tries to block users who dont send it, e.g., Cloudflare users

Thus, before one even considers all the other behaviour of this website operator, some of which is mentioned in this thread, there is a huge red flag for anyone who pays attention to EDNS subnet

As with almost all websites repeated DNS lookups are not an absolute requirement for successful HTTP requests

There are some IP addresses for archive.{today,is,md,ph,li,...} that have continued to work for years

I use archive.today all the time. How do you access pages, like for instance on the economist, without it?

> Whomever it is, they have lot of info about HN users' reading habits given that archive.today URLs are so heavily promoted by HN submitters, commenters and moderators

Anyone interested in the reading habits of HN users can just take a look at news.ycombinator.com ;)

> Whomever it is, they have lot of info about HN users' reading habits given that archive.today URLs are so heavily promoted by HN submitters, commenters and moderators

It's not promoted, it's just used as a paywall bypass so everyone can read the linked article.

you can change the tld of any archive.today link if .today doesn't work. for example archive.ph, archive.is, archive.md, etc

The fact is i cant have a discussion about a paywalled article without reading it. Archive.today is popular as a paywall bypass because nobody wants HN to devolve into debate based on a headline where nobody has rtfa.

"archive.today" as used here means the collection of archive.tld domains, where .tld could be ".is", ".md", ".ph", etc.

"promoted" as used here means placing an archive.tld URL at the top of an HN thread so that many HN readers will follow it, or placing these URLs elsewhere in threads

>I can't use archive.today. I tried but gave up. Too many hassles.

What hassles have you experienced?

I use the Archive Page[0] extension which is really easy to use.

The only thing that annoys me about it is the repeated requests (starting about eight or nine months ago) to complete CAPTCHAs.

[0] https://addons.mozilla.org/en-US/firefox/addon/archive-page/

I don't know, but archive sites could at least publish hashes of the content at archive time. This could be used to prove an archive wasn't tampered with later. I'm pretty underwhelmed by the Wayback Machine (archive.org), it's no better technically than archive.today.

Unfortunately you can't usefully replay TLS and be able to validate it, so no that does not work. Best strategy would probably be a public transparency log, but websites are pretty variable and dynamic so this would be unlikely to work for many.

[dead]

I'm wondering the same thing, would be great to have something similar for personal use

P2p is generally bad for this usecase. P2P generally only works for keeping popular content around (content gets dropped when the last peer that cares disconnects). If the content was popular it wouldnt need to be archived in the first place.

IMO there is actually a very low hanging fruit here, even without P2P or DHTs we could have an URI scheme that consists of a domain and document hash. It is then up to the user to add alternate mirrors for domains. Aside from privacy, it doesn't really matter who answers these requests since the documents are self-signing.

> I figured that they have found an (automated) way to imitate Googlebot really well.

If a site (or the WAF in front of it) knows what it's doing then you'll never be able to pass as Googlebot, period, because the canonical verification method is a DNS lookup dance which can only succeed if the request came from one of Googlebots dedicated IP addresses. Bingbot is the same.

> I've seen people claiming that they have a bunch of paid accounts that they use to fetch the pages, which is, of course, ridiculous.

The curious part is that they allow web scraping arbitrary pages on demand. So if a publisher could put in a lot of arbitrary requests to archive their own pages and see them all coming from a single account or small subset of accounts.

I hope they haven't been stealing cookies from actual users through a botnet or something.

I’m an outsider with experience building crawlers. You can get pretty far with residential proxies and browser fingerprint optimization. Most of the b-tier publishers use RBC and heuristics that can be “worked around” with moderate effort.

> which is, of course, ridiculous.

Why? in the world of web scrapping this is pretty common.

It's because it's actively maintained, and bypassing the paywalls is its whole selling point, thus, they do have to be good at it.

They bypass the rendering issues by "altering" the webpages. It's not uncommon to archive a page, and see nothing because of the paywalls; but then later on, the same page is silently fixed. They have a Tumblr where you can ask them questions; at one point, it's been quite common for everyone to ask them to fix random specific pages, which they did promptly.

Honestly, you cannot archive a modern page, unless you alter it. Yet they're now being attacked under the pretence of "altering" webpages, but that's never been a secret, and it's technologically impossible to archive without altering.

I imagine accounts are the only way that archive.today works on sites like 404media.co that seem to have server sided paywalls. Similarly, twitter has a completely server sided paywall.

It’s not reliable, in the sense that there are many paywalled sites that it’s unable to archive.

There was some beef the site owner had with Cloudflare where if your were using Cloudflare DNS it wouldn’t serve anything to you? Is that still happening?

Not sure why it would only be on archive.is and not the others but ‘is’ loads for me.

See https://en.wikipedia.org/wiki/Wikipedia%3ARequests_for_comme...

They've changed usernames they use to post under. That's the only "altered" allegation they've been accused of.

BTW, they also alter paywalls and other elements, because otherwise, many websites won't show the main content these days.

It kind of seems like "altered" is the new "hacker" today?

It's a reminder how fragile and tenuous are the connections between our browser/client outlays, our societal perceptions of online norms, and our laws.

We live at a moment where it's trivially easy to frame possession of an unsavory (or even illegal) number on another person's storage media, without that person even realizing (and possibly, with some WebRTC craftiness and social engineering, even get them to pass on the taboo payload to others).

I mean, the admin of archive.today might face jail time if deanonymised, kind of understandable he's nervous. Meanwhile for Patokallio it's just curiosity and clicks

That was private negotiations, btw, not public statements.

In response to J.P's blog already framed AT as project grown from a carding forum + pushed his speculations onto ArsTechnica, whose parent company just destroyed 12ft and is on to a new victim. The story is full of untold conflicts of interests covered with soap opera around DDoS.

Just curious: is this of any significance?

I prefer archive.today because the Internet Archive’s Wayback Machine allows retrospective removals of archived pages. If a URL has already been crawled and archived, the site owner can later add that URL to robots.txt and request a re-crawl. Once the crawler detects the updated robots.txt, previously stored snapshots of that page can become inaccessible, even if they were captured before the rule was added.

Unfortunately this happens more often than one would expect.

I found this out when I preserved my very first homepage I made as a child on a free hosting service. I archived it on archive.org, and thought it would stay there forever. Then, in 2017 the free host changed the robots.txt, closed all services, and my treasured memory was forever gone from the internet. ;(

The archive.today operator claims on his blog that this was nothing major: https://lj.rossia.org/users/archive_today/

> Regarding the FBI’s request, my understanding is that they were seeking some form of offline action from us — anything from a witness statement (“Yes, this page was saved at such-and-such a time, and no one has accessed or modified it since”) to operational work involving a specific group of users. These users are not necessarily associates of Epstein; among our users who are particularly wary of the FBI, there are also less frequently mentioned groups, such as environmental activists or right-to-repair advocates.

> Since no one was physically present in the United States at that time, however, the matter did not progress further.

> You already know who turned this request into a full-blown panic about “the FBI accusing the archive and preparing to confiscate everything.”

Not sure who he's talking about there.

>> an analysis of existing links has shown that most of its uses can be replaced.

>Oh? Do tell!

They do. In the very next paragraph in fact:

   The guidance says editors can remove Archive.today links when the original 
   source is still online and has identical content; replace the archive link so 
   it points to a different archive site, like the Internet Archive, 
   Ghostarchive, or Megalodon; or “change the original source to something that 
   doesn’t need an archive (e.g., a source that was printed on paper)

I would be suprised if archive.today had something that was not in the wayback machine

> the community should figure out how to efficiently remove links to archive.today

You're part of the community! Prove him right!

> i don't know anything specific about the site or any conflicts involved, yet this smells like a negative PR campaign to me...

What possible value could a comment from someone who has no knowledge of the site or conflict add to this discussion?

I know I'm arguing with a bot that nobody monitors, but it's already in the fucking post.

I see them everywhere where paywalled content is referenced

Shows that it's a great archival service if the most censored people are able to use it without their archives being censored.

Wayback machine won't bypass paywall nor pirate content, not to mention they are under US jurisdiction. You can't have your cake and eat it.

The objection is to this specific archieve service not archiving in general.

Wikipedia shouldn't allow links to sites which intentionally falsify archived pages and use their visitors to perform DDOS attacks.

They generally do. Random example, citation 349 on the page of George Washington: ""A Brief History of GW"[link]. GW Libraries. Archived[link] from the original on September 14, 2019. Retrieved August 19, 2019."

If you read the linked article it is discussed

Why? The decision seems reasonable at first sight.

About how much had you previously donated over the years?

[flagged]

[flagged]