Comment by awesome_dude
5 hours ago
> Even banking works this way. All database books have the usual “you can’t debit twice, so you need transactions”…bullshit. But think of a money transfer across banks and possibly across countries? Not globally atomic..
Banking is my "go to" anology when it comes to eventual consistency because 1: We use banking almost universally the same ways, and 2: we understand fully the eventual consistency employed (even though we don't think about it)
Allow me to elaborate.
When I was younger we had "cheque books" which meant that I could write a cheque (or check if you're American) and give it to someone in lieu of cash, they would take the cheque to the bank, and, after a period of time their bank would deposit funds into their account, and my bank would debit funds from mine - that delay is eventual consistency.
That /style/ of banking might have gone for some people, but the principle remains the same, this very second my bank account is showing me two "balances", the "current" balance and the "available" balance. Those two numbers are not equal, but they will /eventually/ be consistent.
The reason that they are not consistent is because I have used my debit card, which is really a credit arrangement that my bank has negotiated with Visa or Mastercard, etc. Whereby I have paid for some goods/services with my debit card, Visa has guaranteed the merchant that they will be paid (with some exceptions) and Visa have placed a hold on the balance of my account for the amount.
At some point - it might be overnight, it might be in a few days, there will be a reconciliation where actual money will be paid by my bank to Visa to settle the account, and Visa will pay the merchant's bank some money to settle the debt.
Once that reconciliation takes place to everyone's satisfaction, my account balances will be consistent.
I have been working on payment systems and it seems that in almost all discussions about transactions, people talk about toy versions of bank transactions that have very little to do with what actually happens.
You don't even need to talk about credit cards to have multiple kinds of accounts (internal bank accounts for payment settlement etc.), multiple involved systems, batch processes, reconciliation etc. Having a single atomic database transaction is not realistic at all.
On the other hand, the toy transaction example might be useful for people to understand basic concepts of transactions.
I don't have a lot of payment experience, but AFAIK actual payment systems work in an append-only fashion, which makes concurrency management easier since you're just adding a new row with (timestamp, from, to, value, currency, status) or something similar. However, how can you efficiently check for overdrafts in this model? You'd have to periodically sum up transactions to find the sender's balance and compare it to a known threshold.
Is this how things are usually done in your business domain?
And then they take that toy transaction model and think that they're on ACID when they're not.
Are you stepping out of SQL to write application logic? You probably broke ACID. Begin a transaction, read a value (n), do a calculation (n+1), write it back and commit: The DB cannot see that you did (+1). All it knows is that you're trying to write a 6. If someone else wrote a 6 or a 7 in the meantime, then your transaction may have 'meant' (+0) or (-1).
Same problem when running on reduced isolation level (which you probably are). If you do two reads in your 'transaction', the first read can be at state 1, and the second read can be at state 2.
I think more conversations about the single "fully consistent" db approach should start with it not being fit-for-purpose - even without considering that it can't address soft-modification (which you should recognise as a need immediately whenever someone brings up soft-delete) or two-generals (i.e. consistency with a partner - you and VISA don't live in the same MySql instance, do you? Or to put it in moron words - partitions between your DB and VISA's DB "don't happen often" (they happen always!))
The point is to give people who don't realise that they have been dealing with eventual consistency all along, that it's right there, in their lives, and they already understand it.
You're right I go into too much detail (maybe I got carried away with the HN audience :-) and you are right that multiple accounts is something else that people generally already understand and demonstrate further eventual consistency principles.
I wasn't criticizing you, just making the point that when people talk about toy example bank transactions, they usually want to just introduce the basic understanding. And I think it ok, but I would prefer that they would also mention that REALLY the operations are complex.
I modified my comment above that by multiple types of accounts I meant that banks have various accounts for settlements with the other banks etc. even in the common payment case.
No, this is confusing how the financial institutions operate as a business with how the data store that backs those institutions operates as a technology.
You can certainly operate your financial system with a double entry register and delayed reconciliation due to the use of credit and the nature of various forms of promissory notes, but you're going to want the data store behind the scenes to be fully consistent with recording those transactions regardless of how long they might take to reconcile. If you don't know that your register is consistent, what are you even reconciling against?
What you're arguing is akin to arguing that because computers store data in volatile RAM and that data will often differ from what is on disk, that you shouldn't have to worry about file system consistency or the integrity of private address spaces. After all, they aren't going to match anyways.
No.
I clearly state
> analogy (sorry about the initial misspell) when it comes to eventual consistency because 1: We use banking almost universally the same ways, and 2: we understand fully the eventual consistency employed (even though we don't think about it)
The point is, you understand that your bank account is eventually consistent, and I have given an explanation of instances of eventual consistency that you already usually know and understand.
You make the mistake of thinking about something else (the back end storage, the double entry bookeeping).