Comment by Gigachad
7 hours ago
If your website itself is advertising a product or service you sell you would still want LLMs to see and fetch it. If you are a news site, blog, or any other website that doesn’t exist to sell something, you are only harmed by ai agents.
In those situations you wouldn't have ads on the human version of the site either, surely?
Sure, if it’s paywalled. Web hosting isn’t free