Comment by mabbo
2 years ago
Round robin is hard when you're doing things concurrently and distributed, because state is hard.
A server is taking requests 30 at a time in separate threads. To add a sequential step for picking which target is next, you need to add a synchronous piece of code that all of the threads depend on- a bottleneck. That's doable, but it adds complexity.
And then you have (let's say) 30 servers making calls. Do they each use round robin load balancing separately, or do they try to coordinate? That's difficult.
"Pick two randomly" requires no state. Every thread and server can run the same algorithm without regard for each other.
> "Pick two randomly" requires no state. Every thread and server can run the same algorithm without regard for each other.
That’s not quite true. Picking between two “buckets” still requires knowing how many “balls” are in each which is state. That state can be local to each server or global, that state can be accessed concurrently or synchronously, but you still have the same problem to solve.
In addition, there is a physical machine somewhere that has all the actual data and it's costly to move. The machine you're connecting to can get taken down by a noisy neighbor and rebalancing data is expensive.