Comment by Szpadel

In our case this was very heavy specialized endpoint and because each request used different set of parameters could not benefit from caching (actually in this case it thrashed caches with useless entries).

This resulted in upscale. When handling such bot cost more than rest of the users and bots, that's an issue. Especially for our customers with smaller traffic.

This request rate varied from site to site, but it ranged from half to 75% of whole traffic and was basically saturating many servers for days if not blocked.