Comment by lmm
3 years ago
> Skype used postgres as queue with a small plugin to process all their CDR many years ago. I have no idea if it used these days but it was 'web scale', 10 years ago. Just working, while people on the internet argued about using a database as a queue is an anti-pattern.
It works great until it doesn't, and the way it breaks puts you in a state that's very difficult to recover from. And if your excuse for using a database as a queue was that you were already running a database, that cuts both ways: congratulations, your queue mess has now brought down your primary datastore too.
> It works great until it doesn't, and the way it breaks puts you in a state that's very difficult to recover from.
This is similar to saying, 'if I mess up all the tables in one database I wreck the rest'. Just my opinion, but this is not actually a thing in databases.
Maybe compromised the performance of one database due to another loading things up? I think database are developed with this as an important consideration. I can't say I have seen this, but admit it's a possibility.
Also, if you run one postgres, you won't have of an issue running another if you have the experience in production.
> This is similar to saying, 'if I mess up all the tables in one database I wreck the rest'. Just my opinion, but this is not actually a thing in databases.
If you mess up the tables in one database it doesn't affect others, but if you lock up the server where it can't respond to queries, that affects every database running on that server.
> Also, if you run one postgres, you won't have of an issue running another if you have the experience in production.
We're talking about using a different feature that you presumably haven't used before, so you won't necessarily know about the admin side of that.
> If you mess up the tables in one database it doesn't affect others, but if you lock up the server where it can't respond to queries, that affects every database running on that server.
How is it different from: putting multiple queues on same redis, when one queue is locked up, others queue are affected?
If that's a real risk, you can always put them into different instances. The solution is exactly the same for redis or postgresql
1 reply →
I wonder if it would be possible to have more than just one PostgreSQL database.
It is, but it comes with a lot of the same costs as having a PostgreSQL database and a proper queue system.
No, you already know how to run and manager a postgres database.
If have you large teams, for sure, pick a proper queue that someone in the team knows well.
2 replies →
> congratulations, your queue mess has now brought down your primary datastore too.
Just don't put your queue tables/logic in the same DB instance as your datastore. There are still a lot of benefits to using the same tech even if you have segregated instances.