Comment by Onavo
7 hours ago
I never really saw the point of Anubis. It feels like a anti DDoS tool that's being repurposed for web scraping.
7 hours ago
I never really saw the point of Anubis. It feels like a anti DDoS tool that's being repurposed for web scraping.
It was always a DDoS tool. AI scrapers just send so many requests that it's effectively a permanent DDoS if your server is not smart enough to reject them or fast enough to serve them all.
Artists ideologically opposed to AI are not the intended audience, they picked it up to show everyone they really mean what they say.
It is an anti-DDoS tool in practice. It became popular precisely because the newest scrapers were so aggressive they were DDoSing their targets.
Anubis is the only anti-"rude corporate http requests amounts" that doesn't only rely on cutting edge javascript VM features. Simple mitigations like the HTML header meta-refresh proof-of-work(waiting) work in all browsers going back 20 years. Rather than just supporting browsers from the last 3~ months like cloudflare.
Important to note that it only helps against DDoS that originates within layer7 e.g. recursively downloading specific page URLs, especially many dynamic ones that are easy to slow down the entire webserver with, such as with git platforms.
Someone (or their botnet) sending a "real" DDoS of a few terabits/sec of layer3 traffic your way cannot be stopped by software running at a higher layer at all. Your only recourse there is playing a cat-and-mouse game of who can change IP addresses fast enough.