Comment by vpShane
7 months ago
Very neat idea to charge the crawlers via headers.
Wondering if it blocks things such as Infatica as well, where AI scrapers use that service to scrape data behind residential proxies, in which it's impossible to get the actual residential proxy for.
App developers hide Infatica drone code in things; I haven't thought of a reasonable way to block it.
IP addresses use the systems, humans use them in certain ways, Cloudflare must have a decent way to block these.
No comments yet
Contribute on Hacker News ↗