Comment by cpursley
2 days ago
Right, plus there's character limitations (column size). This is why I prefer listening to the Postgres WAL for database changes:
https://github.com/cpursley/walex?tab=readme-ov-file#walex (there's a few useful links in here)
I found recently that you can write directly to the WAL with transactional guarantees, without writing to an actual table. This sounds like it would be amazing for queue/outbox purposes, as the normal approaches of actually inserting data in a table cause a lot of resource usage (autovacuum is a major concern for these use cases).
Can’t find the function that does that, and I’ve not seen it used in the wild yet, idk if there’s gotchas
Edit: found it, it’s pg_logical_emit_message
pg_logical_emit_message() is how I recommend users on Postgres to implement the outbox pattern [1]. No table overhead as you say, no need for housekeeping, etc. It has some other cool applications, e.g. providing application-specific metadata for CDC streams or transactional logging, wrote about it at [2] a while ago. Another one is making sure replication slots can advance also if there's no traffic in the database they monitor [3].
[1] https://speakerdeck.com/gunnarmorling/ins-and-outs-of-the-ou...
[2] https://www.infoq.com/articles/wonders-of-postgres-logical-d...
[3] https://www.morling.dev/blog/mastering-postgres-replication-...
You know, this would be a great talk at the 2026 Carolina Code Conference...
Ha, your [2] is how I learnt about it! Thanks :)
`pg_logical_emit_message()` is great and better than `NOTIFY` in terms of how it works, but...
`pg_logical_emit_message()` perpetuates/continues the lack of authz around `NOTIFY`.
What do you mean by this? What authz would you expect/like?
1 reply →
One annoying thing is that there is no counterpart for an operation to wait and read data from WAL. You can poll it using pg_logical_slot_get_binary_changes, but it returns immediately.
It'd be nice to have a method that would block for N seconds waiting for a new entry.
You can also use a streaming replication connection, but it often is not enabled by default.
I think replication is the way to go, it’s kinda what it’s for.
Might be a bit tricky to get debezium to decode the logical event, not sure
2 replies →
For node.js users there is postgres.js that can listen to the Postgres WAL and emit node events that can be handled by application code.
Yeah until vendors butcher Postgres replication behaviors and prevent common paths of integrating these capabilities into other tools. Looking at you AWS