Comment by hombre_fatal
1 day ago
Botnets don't even use their own hardware.
Why would someone renting dirt cheap botnet time care if the requests take a few seconds longer to your site?
Plus, the requests are still getting through after waiting a few seconds, so it does nothing for the website operator and just burns battery for legit users.
If you're a botnet operator of a botnet that normally scraped a few dozen pages per second and then noticed a site suddenly taking multiple seconds per page, that's at least an order of magnitude (or two) decrease in performance. If you care at all about your efficiency, you step in and put that site on your blacklist.
Even if the bot owner doesn't watch (or care) about about their crawling metrics, at least the botnot is not DDoSing the site in the meantime.
This is essentially a client-side tarpit, which are actually pretty effective against all forms of bot traffic while not impacting legitimate users very much if at all.
A tarpit is selective. You throw bad clients in the tarpit.
This is something you throw everyone through. both your abusive clients (running on stolen or datacenter hardware) and your real clients (running on battery-powered laptops and phones). More like a tar-checkpoint.
There is still an opportunity cost. They can scrape just your site or they can scrape 100 other sites without POW (no idea if it is 10, 100 etc)
Websites aren't really fungible like that, and where they are (like general search indexing for example), that's usually the least hostile sort of automated traffic. But if that's all you care about, I'll cede the point.
Usually if you're going to go through the trouble of integrating a captcha, you want to protect against targeted attacks like a forum spammer where you don't want to let the abusive requests through at all, not just let it through after 5000ms.
So it's the same as a sleep()
No, because the bot can just also sleep and scrape other sources in that time. With pow, you waste their CPU cycles and block them from doing other work.
Botnets just shift the bottleneck from "how much compute can they afford to buy legit" to "how many machines can they compromise or afford to buy on the black market". Either way it's a finite resource, so making each abusive request >10,000x more expensive still severely limits how much damage they can do, especially when a lot of botnet nodes are IoT junk with barely any CPU power to speak of.