Comment by crote

Being indexed in a search engine benefits the website owner, as it drives traffic towards the website.

Being used as AI training data provides negative value for a website owner, as it takes traffic away.

It's the difference between a movie review, and a ripped torrent.