Comment by Capricorn2481
19 days ago
I don't understand this. You don't have routes your users might need in robots.txt? This article is about bots accessing resources that other might use.
19 days ago
I don't understand this. You don't have routes your users might need in robots.txt? This article is about bots accessing resources that other might use.
It seems better to put fake honeypot urls in robots.txt, and block any up that accesses those.
Blocking will never work.
You need to impose cost. Set up QoS buckets, slow suspect connections down dramatically (almost to the point of timeout).
Ah I see