Comment by altairprime
2 days ago
As things stand right now, the current web may not be possible to maintain for old and low-end phones given the costs imposed by AI training crawlers.
Please propose an alternative to both Cloudflare and Anubis, that shields websites against inhuman traffic without frequent operator intervention (or otherwise negates the capacity costs they pay for AI crawling) and is compatible with low-end smartphones.
Certainly, I imagine Anubis would be interested in adopting it if it’s effective!
The solution is building simpler and more efficient websites. You won't win the arms race, so make it irrelevant. Any random VPS hosting a static website can already serve millions of requests an hour.
Great you now made the internet unusable for half the planet.
AI is absolutely going to make the internet unusable for half the planet if we don't find a better way; it's happening in realtime and I hate to see it. I would love to know of a better way, but everyone's so busy clamoring against effective solutions without proposing anything else that would work.
Doing nothing doesn't work. Doing Cloudflare isn't acceptable. Doing Anubis makes the Internet unusable "for half the planet". I'm on your side morally, but site operators can't afford to be idealists in the face of AI crawling bills.
What do you, does anyone, suggest that hasn't been tried, or adopted, or considered? What's left that will defend operators against AI's traffic flood that isn't eating a thousand doller crawler bill, paying Cloudflare, or locking out half the planet?
Sounds like something the government should regulate and own then rather than relying on corporate benefactors that all profit off of malware advertising.
That will be about as effective as attempts to block soccer streams in Spain are, and I don't think it'll pan out as a viable solution without worldwide enforcement backed by BGP bans of providers that continue to host non-compliant sites after notified. It's technically possible to consider a future where RIPE might withdraw your BGP assignment if you host advertising malware on your website, but ultimately that's just as dystopic as the DMCA is today, only scaled up to worldwide impacts and drama.