Comment by hirsin

If you're claiming that the operators of high volume AI scrapers that wantonly disregard rate limits and all common sense are unethical then I'm right there with you. But that's not at all what was described upthread nor is it the only way in which bots get used by any stretch of the imagination.

As far as anti-bot countermeasures go I quite like proof of work solutions since those disproportionately impact high volume scrapers without noticeably impeding a small hobby project.

Unfortunately the operators of many major websites appear to want something akin to DRM with the excuse of bots used merely as window dressing.

There was a time when a person could walk through a few department stores every week (or even every day) just to take note of some prices along the way, and ultimately tabulate them to try to identify and snatch up the best deal once it happens.

And if everyone did this, it'd be a real problem. The stores would be clogged up by geeks writing notes in little books with Parker Jotters and just basically wasting space and taking up air conditioning while they sleuth out the best way to put the screws to the company for a few measly dollars.

That'd be awful.

But not many people ever did that in stores, and not many individual people are doing that today with the web. It's really not a problem.

(And if a website in 2026 can't stand the burn of several thousand personal scrapers that are operated by people who actually want to buy stuff from it, then maybe that system simply sucks and needs to be rethought.)