Comment by rvz
8 hours ago
I am going to assume that this also destroys millions of AI agents and bot scrapers this time which is why some “AI Engineers” were complaining about this recently.
Well, this is how Google will kill all the scrapers on its search data.
Not entirely, Google's own page says:
> Fraud Defense leverages a sophisticated and adaptable risk analysis engine to shield against automated software. It is specifically designed to orchestrate trust for the agentic web, neutralizing malicious scrapers while welcoming legitimate AI agents.
I'm sure it'll block a whole bunch of awful scrapers but if Google doesn't hate a bot, it'll be able to pass.
Sounds like an anti-competitive play to build an AI moat. They'll introduce a sham "verification program" and only allow bots operated by entities large enough to sue them for anti-competitive conduct.
Does mass scraping need google for content discovery? Surely most sites contain a site map or index that would effectively self enumerate once you know the domain, which is more often than not publicly disclosed?
What matters is when websites put this new version of reCAPTCHA on their site, just like archive.is has done. Then the scrapers will have a hard time getting around that.