Comment by miki123211
2 months ago
No, but AFAIK they pulled some shenanigans with "bundling" Gemini scraping and search engine scraping.
Almost everybody wants to appear in search, so disallowing the entirety of Google is far more costly than E.G. disallowing Openai, who even differentiates between content scraped for training and content accessed to respond to a user request.
While there isn't a way to differentiate between scraping for training data and content accessed in response to a user request, I think you can block Googlebot-extended to block training access.