Comment by jpc0
1 day ago
“Normal load balancing” means “Request A goes to server A”, “Request B goes to server B” and there is no state held in the server, if there is a session its stored in a KV store or database which persists.
With SSE the server has to be stateful, for load balancing to work you need to be able to migrate connections between servers. Some proxies / load balancers don’t like long lasting connections and will tear them down if there has been no traffic so your need to constantly send a heart beat.
I have deployed SSE, I love the technology, I wouldn’t deploy it if I don’t control the end devices and everything in between, I would just do long polling.
This take that SSE is stateful is so strange. Server dies it reconnects to another server automatically (and no you don't need ping/pong). It's only stateful if you make it stateful. It works with load balancing the same as anything else.
The SSE spec has an event id and the spec states sending last event id on reconnection. That is by its nature stateful, now you could store that in a DB/KV itself but presumably you are already storing session data for auth and rate limiting so now you had to implement a different store for events.
And I too naively believed there won’t be a need for ping/pong, then my code hit the real world and ping/pong with aliveness checks was in the very next commit because not only do load balancers and proxies decide to kill your connection, they will do it without actually closing the socket for some timeout so your server and client is still blissfully unaware the connection is dead. This may be a bug, but it’s in some random device on the internet which means I have to work around it.
Long polling might run into the same issues but in my experience it hasn’t.
I really do encourage you to actually implement this kind of pattern in production for a reasonable number of users and time, there’s a reason so many people recommend just using long polling.
This also assumes long running servers, long polling would fall back to just boring old polling, SSE would be more expensive if your architecture involves “serverless”.
Realistically I still have SSE in production, on networks I can control all the devices in the chain because otherwise things just randomly break…
Your description of "normal load balancing" is certainly one way to do load balancing, but in no way is it the presumptive default. Keeping session data in a shared source of truth like a KV store or DB, and expecting (stateless) application servers to do all their session stuff thru that single source of truth, is a fine approach for some use cases, but certainly not a general-purpose solution.
> With SSE the server has to be stateful, for load balancing to work you need to be able to migrate connections between servers.
Weird take. SSE is inherently stateful, sure, in the sense that it generally expects there to be a single long-lived connection between the client and the server, thru which events are emitted. Purpose of that being that it's a more efficient way to stream data from server to client -- for specific use cases -- than having the client long-poll on an endpoint.
> Keeping session data in a shared source of truth like a KV store or DB, and expecting (stateless) application servers to do all their session stuff thru that single source of truth
What would be a scalable alternative?
Simple edge-case why this is a reasonable approach. Load balancer sends request to server A, server A sends response and goes offline, now load balancer has to send all request to server B->Z until server A comes back online. If the session data was stored on server A all users who were previously communicating to server A now lost their session data, probably reprompting a sign-in etc
Theres some state you can store in a cookie, hopefully said state isn’t in any was mean to be trusted since rule 1 of web is you don’t trust the client. Simple case of a JWT for auth, you still need to validate the JWT is issued by you and hasn’t been invalidated, ie a DB lookup.
This is the same with request response. You need to auth on each request (unless you use a cookie).
1 reply →