Comment by hmaxdml
4 days ago
Not saying this in a mean way:
It seems to me this can be boiled down to "using things without understanding how they work doesn't scale". Yes, vanilla listen/notify doesn't scale. But the OP actually figured out how to make it scale. So your engineers don't have to.
As the community builds, over time, distributed systems, it also understands which brick can do what, and it turns out that starting with less bricks and adding some when you actually need them makes for healthier systems.
Nah this isn't mean at all. I think this is the correct takeaway. I actually felt like I understood how the system worked and it wasn't that difficult for me, but I also understood how, as we were adding engineers who were all very under water, the idea of learning our queueing system to modify a core feature it powered seemed like a big mental context switch (bigger than it actually was). I noted this in another comment, but I am likely discounting how big of an effect our scale/growth impacted peoples' ability to adapt this system. That said, I will pretty much always choose the simple/dumb/easy to understand thing from the start even if it adds another moving component.
Also the results given it the article are still worse than redis with zero work and no risk of breaking later, both in nb of req and latency.
And you can move redis to another server next later if needed to split the db and notification load.
[flagged]