Comment by bodantogat
1 day ago
I feel the pain — it’s very difficult to detect many of the less ethical scrapers. They use residential IP pools, rotate IPs, and provide valid user agents.
1 day ago
I feel the pain — it’s very difficult to detect many of the less ethical scrapers. They use residential IP pools, rotate IPs, and provide valid user agents.
> residential IP pools
So, is this a new profit center for sleazeball household ISPs?
No, this is done by paying app developers to bundle some random SDK. Search for Bright Data.
"Residential proxy" is just a word for a botnet. Apps and programs come with a trojan built in that offers your device as an exit node to "monetize" their apps.
It "residential proxy" sounds a lot better when you're talking to VC investors, though.
"free" vpns do this.
And many proxy/scraping providers now use real browsers that can automatically bypass cloudflare captchas as well, and a bot with a real browser similarly won't be clicking on any invisible links, so... I am skeptical just how long this will make an appreciable difference.
So what you’re saying is that a CAPTCHA cannot actually T C and H A.
Telling humans and computers apart was never the purpose of CAPTCHAs, only how they initially worked. The name has been a complete misnomer for at least a decade now. Its actual purpose is, and has always been, abuse prevention. Has it been successful? Some yes, some no, and a lot of collateral damage. Its mode of operation now looks a lot like inscrutable blacklisting for some plus inconvenience and bad rate limiting for the rest.
6 replies →