Comment by tekacs
2 days ago
I mean that they're telling developers that they should use Cloudflare's platform to build agents, the kind of agents that would go across the web and act on behalf of users... but then they're also the ones blocking those requests.
This always engenders a solid amount of distaste from me, because much like Google and Chrome, it creates the incentive for you to treat yourself better than others. Especially coupled with the trust stuff. Of course, Cloudflare is always going to trust their own platform.
There are ways to use those agents ethically. Cloudflare isnt forcing you to be a dick with your agents. Infact the blocking side encourages you to use them ethically. I dont see the problem.
I thought the issue people have with AI scrapers is the ones that DDoS sites to scrape everything for training purposes, rather than the ones that interactively query specific content to support an active conversation?
If you read the article, you'll notice that they're explicitly making sure to block the ones that interactively query specific content too.
First one is problem because it brings no traffic back and occasionally DDoSes the site
Second one is problem because it DDoSes the site. "Just" queries for hundred thousand people (if you happened to be good source for that bit of knowledge) that don't bring actual people to your site is also a problem
First one is not occasional, it is the anonymous global adversary probably ddosing your site right now. It never relents.