Comment by Faaak

Depends on the website though. I want LLMs to scrap my B2B website, because then it's shown to the user and they will likely use my product afterwards