Comment by SchemaLoad

1 day ago

Cloudflare has a service for this now that will detect AI scrapers and send them to a tarpit of infinite AI generated nonsense pages.

Wow, so to prevent AI scrapers from harvesting my data I need to send all of my traffic through a third party company that gets to decide who gets to view my content. Great idea!

  • You don’t need to do anything. You can use any number of solutions or roll your own.

    Someone shared an alternative. Must everything in AI threads be so negative and condescending?

    • Yes, they could roll their own, but you have no issues with this being necessary? I think the attitude of "just deal with it" is far more negative than someone expressing they are upset with the state of the internet, its controllers, and its abusers.

      2 replies →

    • > Must everything in AI threads be so negative and condescending?

      Because if I own a website or a service and it is being degraded or slowed by some third party tool that wants to slurp its content for his own profit and don't even share, I tend to be irritated. And AI apologists/evangelists don't help when they try to justify the behavior.

Unfortunately, Cloudflare often destroys the experience for users with shared connections, VPNs, exotic browsers… I had to remove it from my site after too many complaints.

Savvy move by cloudflare, once they have enough sites behind their service they can charge the AI companies to access their cached copies on a back channel

cloudflare also blocks my human-is-driving browser all the time

"enahble javascript and cookies to continue"

also unsupported browser

Modern scrapers are using headless chromium which will not see the invisible links, so I'm not sure how long this will be effective.