Comment by phyzome

I can't imagine IA ever going for it. You'd need a separate org that just scrapes for AI training, because its bot is going to be blocked by anyone who is anti-AI. It wouldn't make sense for it to serve multiple purposes.

Common Crawl would be a better fit, but still might not want to serve in that capacity.