Comment by Terretta
2 years ago
> what are the advantages against taking the least occupied server/bin always
The other answers are saying for algo reasons. Sure, but, pick two is even more better for uptime reasons.
In serving at scale, quite often the "least occupied" server of all of them is one with an error causing it to have least load/connections/whatever.
By starting with a random two, you pick among probably healthy servers, and then the least load.
> good improvements with nginx load balancing when changing its configuration to least_conn, that chooses the server with least connections.
nginx doesn't know why your app is failing to keep connections open long enough to serve them, least_conn between two random results in far fewer 100% "outages" when a single box or subset of boxes among 10s or 1000s is bad.
Wouldn't that be better solved with health checking and removing the poor performing server from the pool?