Comment by marginalia_nu
11 days ago
Sadly they seem to be getting through this one lately. Had a scraper hitting me with 80 qps punching straight through Anubis. Had to set up a global rate limit that browned out the functionality they were interested in[1] in excessive load.
[1] This form https://marginalia-search.com/site/news.ycombinator.com
Probably headless chrome. I'm going to investigate.
They were hitting me at an incredible rate sustained load for like a week, seems like a lot of resources if it is indeed headless chrome.
Is there anything I can extract, fingerprint wise, if they come back to work out what's going on?