Comment by mmarian
2 days ago
Some scrapers/scanners use residential IPs. Aren't you worried you'll end up blocking legitimate traffic?
2 days ago
Some scrapers/scanners use residential IPs. Aren't you worried you'll end up blocking legitimate traffic?
Two points:
1. The websites I run get so little traffic it doesn't matter. They're mostly for my own entertainment / experimentation.
2. If they're allowing their IP address to be used for pricing scrapers then I consider that within the blurry definition of malicious anyway.
I don't mind if you disagree with me on point #2, and I grant that if I was running some super popular web service, maybe my tune would change.
I have four tiers of scanning paranoia, so I can ramp up and down if need be (I'm not sure if that's documented in GitHub though...)
Fair enough. Let me know if you end up documenting it in GitHub somewhere, I'm curious what people set up.