Comment by WarOnPrivacy
18 hours ago
>> but the point is that big company crawlers aren’t paying for questionably sourced residential proxies.
> I worked for Microsoft doing malware detection back 10+ years ago, and questionably sourced proxies were well and truly on the table
Big Company Crawlers using questionably sourced proxies - this seems striking. What can you share about it?
They worked on malware detection. The most likely reason is very obvious: if you only allow traffic from residential addresses to your Command & Control server, you make anti-malware research (which is most likely coming from either a datacenter or an office building) an awful lot harder - especially when you give non-residential IPs a different and harmless response instead of straight-up blocking them.
they probably can't because some of the proxies were used by TLAs is my guess...