Comment by im3w1l

If you have a static site with content you want to share broadly, nothing is wrong.

It becomes a problem when it’s used to spam unwanted content faster than your human moderators can come up with.

Someone might bot to scrape your content and repackage it on their own site for profit.

The bots might start interacting with your real users, making them frustrated and driving them away.

Apparently serving HTML + other static content is more expensive than ever, probably because people go the most expensive routes for hosting their content. Then they complain about bots making their websites cost $100/month to host, when they could have thrown up Nginx/Caddy on a $10/month VPS and basically get the same thing, except they would need to learn server maintenance too, so obviously outside the question.

3 reasons basically:

1. non-humans can create much more content than humans. There's a limit to how fast a human can write, a bot is basically unlimited. Without captchas, we'd all drown in a see of Viagra spam, and the misinformation problem would get much worse.

2. Sometimes the website is actually powered by an expensive API, think flight searches for example. Airlines are really unhappy when you have too many searches / bookings that don't result in a purchase, as they don't want to leak their pricing structures to people who will exploit them adversarially. This sounds a bit unethical to some, but regulating this away would actually cause flight prices to go up across the board.

3. One way searches. E.g. a government registry that lets you get the address, phone number and category of a company based on its registration number, but one that doesn't let you get the phone numbers of all bakeries in NYC for marketing purposes. If you make the registry accessible for bots, somebody will inevitably turn it into an SQL table that allows arbitrary queries.