Comment by Root_Denied
6 hours ago
Banking/Finance is the one industry I've seen push back against this type of thinking. Transactions must be handled in a perfect and repeatable way, or the system is unusable as far as the company is concerned.
There's definitely still AI/LLM integration happening, but is kept out of specific areas of the business.
Same with aviation and safety. I honestly believe all programmers should, early in their careers, do a brief "tour of duty" in an industry where the stakes are high and "good enough" isn't good enough. You might not choose to make it your entire career, but at least you're exposed to the discipline, however briefly. Most software developers today have never in their lives worked on a project where defects were taken seriously and where there was process and documentation designed to reduce their occurrence.
To be fair, I'm inclined to believe that most of this pressure to lean (if not rely) on LLMs is not coming from programmers, but rather from middle and upper management: banks are resisting this not because their programmers are holding some kind of line, but because management is not giving-in to the LLM siren call. So that tour of duty, while a good idea, would be more about teaching management skills than anything else.
The normalisation of inexplicable failures is, in my opinion, the consequence of programmers being unable to refuse their managers who have fallen for the LLM siren call, lest they lose their jobs.
I think the failures matter in non-sensitive environments as well albeit with a different threshold.
If you randomly screw up customer orders (think of DoorDash or an online shop or Airbnb). They lose trust in you and you lose your business to the competition. Going happy go lucky and being irresponsible in the business can bankrupt most* businesses.
* well, of course except the criminal empires which are bailed out by our tax money.
I've been seeing the very opposite. Fintech companies treating design of financial systems with the casualness of a frontend aesthetic change. And tons of business people integrating their vibecoded POCs with financially sensitive data sources.
Fintech != banking/finance, at least not necessarily.
I’ve had banking transactions fail a number of times for unknowable or ill-defined reasons. One just last week in fact.
So that is a strange choice for repeatable, understandable operations. Might as well use Jev.
From a consumer perspective there's an unknown number of layers between your actions or instructions and what the backbone tech of the bank is doing, but having worked in the space I can point to a few things.
Firstly, the transaction failed and notified you about the error - that's certainly intentional.
Second, there's failures that would be invisible to you as the customer, such as "instead moving $100 from account A to account B it credited account B but didn't debit account A, without generating an error". Those are mostly the systems I'm talking about being insulated from AI development. Without more detail about the exact problem you had it's hard to tell if it's a failure in customer facing systems or backend infra.
Third, I'm assuming you were able to reach out to the bank directly and resolve the issue by talking to a person (if the issue was really outside the norm), which isn't something you can assume will be possible with a lot of customer service ops these days (or you're going to be waiting hours/days for that callback).
Fourth, you can't really know the error rate of the bank's systems, or how common a given particular error is. It may be a known issue, or it may be a completely unreported one. Assuming it's an error with something on the backend/backbone of the bank's operations, it's running code that can be inspected, reviewed, understood, and fixed - sometimes by a very expensive COBOL consultant.