Comment by mystraline
10 days ago
IP blocking Asia took my abusive scans down 95%.
I also do not have a robots.txt so google doesnt index.
Got some scanners who left a message how to index or dei dex, but was like 3 lines total in my log (thats not abusive).
But yeah, blocking the whole of Asia stopped soooo much of the net-shit.
> I also do not have a robots.txt so google doesnt index.
That doesn't sound right. I don't have robots.txt too but Google indexes everything for me.
https://news.ycombinator.com/item?id=46681454
I think this is a recent change.
All the comments there seem to suggest that there has been no change and that robots.txt isn't required.
If you block the rest you get to 100%.
How did you block Asia, cloudflare or something else?
You can block at your gateway/router. Lots of places have country IP ranges[1], and there are even more or less frequently updated lists of 'malicious' IP ranges[2]. Some gateway providers include 'block by country' and/or 'download blocklists automatically' as a feature.
[1] e.g. https://github.com/ipverse/geo-ip-blocks
[2] e.g. https://github.com/bitwire-it/ipblocklist
You can download weekly IP blocks of regions.
I import them into iptables and wholesale block them all.
I dont deal with eastdakota's pile of shit.