Comment by maxchisto
11 hours ago
I'm suspicious of load shedding not mentioned in the article. Combine that with exp backoff in the caller and you got yourself a pretty robust starting point
11 hours ago
I'm suspicious of load shedding not mentioned in the article. Combine that with exp backoff in the caller and you got yourself a pretty robust starting point
I’d like to mention it since Uber has a really cool load shedder [1], also implemented similarly by Netflix [2] and failsafe-go [3]. It basically looks for points where more requests per second suddenly cause a significant increase in latency and calls that the concurrency limit.
1: https://www.uber.com/us/en/blog/cinnamon-using-century-old-t...
2: https://github.com/Netflix/concurrency-limits
3: https://failsafe-go.dev/adaptive-limiter/
429 (and sometimes 503) errors returned by servers might well be a symptom of intentional load shedding. Perhaps it's just not explicitly called out as a server behavior that induces client retries.