Comment by timpera

2 days ago

Please don't use Anubis, it makes visiting websites very difficult (often multiple minutes wait times) on old hardware and low-end smartphones.

As things stand right now, the current web may not be possible to maintain for old and low-end phones given the costs imposed by AI training crawlers.

Please propose an alternative to both Cloudflare and Anubis, that shields websites against inhuman traffic without frequent operator intervention (or otherwise negates the capacity costs they pay for AI crawling) and is compatible with low-end smartphones.

Certainly, I imagine Anubis would be interested in adopting it if it’s effective!

  • The solution is building simpler and more efficient websites. You won't win the arms race, so make it irrelevant. Any random VPS hosting a static website can already serve millions of requests an hour.

  • Great you now made the internet unusable for half the planet.

    • AI is absolutely going to make the internet unusable for half the planet if we don't find a better way; it's happening in realtime and I hate to see it. I would love to know of a better way, but everyone's so busy clamoring against effective solutions without proposing anything else that would work.

      Doing nothing doesn't work. Doing Cloudflare isn't acceptable. Doing Anubis makes the Internet unusable "for half the planet". I'm on your side morally, but site operators can't afford to be idealists in the face of AI crawling bills.

      What do you, does anyone, suggest that hasn't been tried, or adopted, or considered? What's left that will defend operators against AI's traffic flood that isn't eating a thousand doller crawler bill, paying Cloudflare, or locking out half the planet?

    • Sounds like something the government should regulate and own then rather than relying on corporate benefactors that all profit off of malware advertising.

      1 reply →

Personally I think that's a better outcome than the alternative.

However another option is to support both. Have a challenge page that requires the visitor to select one of several options. Cloudflare could be one of those.

what kind of hardware would take minutes to complete the checks?

  • It really depends on the difficulty level chosen by the website. I'm testing on both a Xiaomi Redmi 5 and my university library's computer.