Comment by chucke
1 month ago
> DBOS has a special @DBOS.Transaction decorator. This runs the entire step inside a Postgres transaction. This guarantees exactly-once execution for databases transactional steps.
I stopped here. I know, authors really want to chase the exactly-once dragon, but this won't scale. If the step takes a long time, it'll keep the transaction open with it for that time. The master replica has to bookkeeping that. That state has to be replicated. That will also affect MVVC further on. As your scale grows, you'll see disk usage growing and eventually swapping, replica lags, AND vacuum halting for surges. I hope your uncalled engineers have a steady supply of coffee.
Yes, that's why the @DBOS.Transaction is just for database operations, which (hopefully!) you've already optimized to not take unreasonable time. These are guaranteed to execute exactly-once. For other, potentially longer-running operations, use @DBOS.step, which is at-least-once. Documentation:
Transactions: https://docs.dbos.dev/python/tutorials/transaction-tutorial
Steps: https://docs.dbos.dev/python/tutorials/step-tutorial
OK, I see I skipped that. Nevertheless, I would expect users to read "exactly once" and put an http call in the step. It's the type of knife people their hands with.
One of the features on our list is to actually warn people when they make external calls inside of a step for this exact reason. You're right, the developer does need to understand idempotency to be successful, but we do everything we can to make sure they are successful without that knowledge.
I, too, found this "exactly-once" language really unclear and a turn-off. What does it actually say that is meaningful and helpful? I can run a transaction in any language and I get, surprise, transactional semantics.
Exactly, I'm surprised that DBOS was founded by the Postgres creator