Comment by 01284a7e
6 days ago
No tests? Just mess up some mundane detail [1] and voila! Wake-up calls and heart attacks for 100,000s of administrators?
1: "Oh, well, this is not a mundane detail, Michael!" https://www.youtube.com/watch?v=3fGHaVn5rGo
There will have been tests, but there will have been missing end-to-end tests. Test 1 will verify that the new system/product emits billing entries in some expected way ("We did 100 bytes of operations and we see we called the billing system for 100 bytes of stuff, yay, test pass"). Test 2 will be in the billing system ("We provide an incoming bill for SKU#12345 for 100 gigabyte-units and we see it costs $17, yay, test passes"). But they won't test the two things together because it will be harder to do and the teams will have different management chains. Seen it happen several times at several companies. Somebody will have said at some point "we should actually have the tests charge money" and somebody else will have said "well we can't have the tests actually charge money, that's a legal/accounting problem, it might even be a crime" and then nobody would have asked what the next best thing was.
> Somebody will have said at some point "we should actually have the tests charge money" and somebody else will have said "well we can't have the tests actually charge money, that's a legal/accounting problem, it might even be a crime" and then nobody would have asked what the next best thing was.
If you support multicurrency billing, then have the tests bill a test customer in XTS (ISO 4217 code for test currency).
Thanks, I wasn’t familiar with XTS.
Now I’m picturing the 3am end of quarter fire drill in finance when they discover the company has accounts receivable of fourteen billion $XTS and it’s appearing in the quarterlies.
You certainly don't want your service caring about a test currency.
Nobody is pricing their services in test currency
They are either pricing in $ or in "units" and then converting upstream
This is an even bigger foot gun than what's happening here
4 replies →
But these aren't the right services where the test should be, right?
There's another service that says "ok we take the 100 bytes from A, and we take the $17 SKU from B, and this should equal $X".
It's the third service that multiplies these things that failed. Where are the tests for that?
To me this sounds like a human-or-LLM-driven error. There must be a pretty limited set of factors that determine a pricing unit: I’m not really sure how a deterministic system could do that infrequently enough to not be a bigger story. Maybe a reeeeaaaallly rare race condition or something like that? To me this smells like having enough manual work involved in the process to fuck something up, but not nearly enough eyes on it to notice.
I’m totally guessing though.
5 replies →
> well we can't have the tests actually charge money, that's a legal/accounting problem, it might even be a crime
You don't have a non-production environment where you can have billing that doesn't involve real money?
the cynical take is of course that the tests were written by Claude and not validated.
Unfortunately "automatic generation" of unit tests have made a big part of the value of unit tests ("I am telling you what I expect here") disappear.
I love how AI means that all human-written code is now perfect, and any problems are onbviously a symptom of vibe coding. Another few years of hearing this and I may begin to believe I only wrote perfect code, when my current memory suggests otherwise.
Nah, this is not uncommon in AWS. I came across a similar bug 10 years ago when playing with Kinesis and nearly had a heart attack at a $2m billing alert! Turns out it was just 5c
[flagged]
Not even tests but just some basic anomaly detection lol.
Like maybe if the bill amounts increase by like 10M% there should be someone that looks into it
You overestimate how much people give shits at big techs like Amazon. When literally everything is driven with sticks instead of carrots, the work culture does not invite employees to proactively care about product quality.
You'd be better off letting the heart attacks happen and take the 3am on-call and be the hero instead. It would be good promo doc material, and being a hero is extremely good insurance against getting kicked out of the country (via the PIP->H1B grace period expiry mechanism).
Speaking from my experience at Amazon this is not the case. Any customer impact like this would necessitate a COE (correction of errors) report, which means a list of required action items to prevent such issues from happening again, which typically suck up at least man-month of labor. Not to mention the report itself, which has to be written by a manager.
In fact, there are regular AWS-wide meetings where L10 technical staff will randomly pick and review reports from across the organization. Getting picked for one of these is not a fun experience.
COEs are such a huge annoyance for teams that they create a strong incentive to be proactive in preventing issues like this from happening. One of the rules when it comes to writing COEs is that they are not the fault of individuals but processes; but in reality, no one wants to be the cause of one.
12 replies →
This ^^^ amplified by indifference and not giving a shit caused by "AI Adoption".
There is literally no fucking reason to try to improve your skill. Any IDIOT with AI will do an OK job.
And no one is shooting for better than OK.
1 reply →
Are you speaking from experience or simply making things up? I know a fair number of former AWS engineers and managers. None of them think like this.
13 replies →
> take the 3am on-call and be the hero instead
Ah yes, the good old ITism "Everything's good, what are we even paying you for?" followed by "Everything's on fire, what are we even paying you for?"
I moved out of it largely for that reason, am now an infrastructure/IT project manager, quite refreshing actually.
4 replies →
While I didn't work on AWS, I did intern on the retail side of Amazon, and there's definitely this sort of monitoring in place. Surely somebody was paged. And even if not, this is "just" the cost explorer estimations, not what is ending up on folk's bills.
I learned about <https://en.wikipedia.org/wiki/2011_T%C5%8Dhoku_earthquake_an...> from alarms like this, as sales in Japan almost entirely stopped.
I've been told a tale of another incident where some customer ran some huge cpu-intensive workload that didn't do any networking. It caused various alarms to fire because it "looked like" a part of the network was idle (potentially indicating some sort of networking failure)
It's generally (in the broad sense) easy to add alarms for things going wrong, but in my experience anomaly detectors are just as likely to fire from other weird things like that happening.
> there's definitely this sort of monitoring in place. Surely somebody was paged.
Well you’re half right. Either there wasn’t monitoring for this or if there was monitoring in place that’s not what caught this, because the page originated from a customer support ticket.
Clearly some folks here have never worked at Amazon before. It's genuinely terrifying to see what most of the internet runs on.
They've already got anomaly detection: their users.
I think billing is the only thing AWS doesn't really care about optimizing or putting enough tests to avoid anomalies lol.
If only there was some way to get anomaly detection services [1] inside of AWS...
1: https://aws.amazon.com/what-is/anomaly-detection/
>no tests?
Earlier this week my slopservant implemented several comprehensive changes to a codebase.
It also wrote extensive tests to verify the correctness of the changes.
A few days later I was working on something else and realized, everything had been implemented backwards, in a way that was nonsensical and also completely pointless.
The many tests it had written were just confirming the LLM's idea of correctness, which turned out to be... completely incorrect.
I laughed when I realized, if I had been using Rust, or indeed, formal verification, that wouldn't have helped at all — it would have just written a mathematical proof, proving the correctness of the wrong thing!
Not sure what lesson to take from that (except read the damn diffs, obviously — it was a hobby project okay ;), but it seems like the more reliable this stuff gets, the more we expect it to work properly, the more risky it becomes.
https://en.wikipedia.org/wiki/Normalization_of_deviance
I'm struggling with this on a hobby project now. I'm torn between continuing the fast pace of development and taking a pause and checking the entire codebase.
The more features I add now will likely make the inevitable refactor much harder, but adding new features is so easy that I want want extend this illusion of productivity just a little bit longer!
Yeah my logic was, we're doing so many refactors that it doesn't make sense for me to start memorizing how things work until things settle down a little.
The argument in favour of "read each delta" (aside from catching them doing stupid shit) is to keep your mental model synced. But I already didn't understand my own code before AI, so that's moot too!
> if I had been using Rust, or indeed, formal verification, that wouldn't have helped at all — it would have just written a mathematical proof, proving the correctness of the wrong thing!
This is a well-known issue in formal methods, often referred to as the "formalization gap". It's pretty much an intractable problem, i.e. all we can do is manage it, not solve it.
In fact this issue exists in many fields, often under the name "specification gap" or "specification problem". You'll find versions of that discussed inv economics, architecture (of buildings, not software), etc. In architecture and construction, a high percentage of mistakes that result in lawsuits turn out to be due not to construction, but to specifications that were incorrect in some way.
The bottom line is that there's no perfect formal/mathematical way to go from intent to proof.
The solution is a skill that explains to the LLM, with a fresh context, to NOT read source files, and to only use XYZ tooling to extract documentation and function signatures, etc. without implementation details.
Then it can write tests based on the interface, and that is usually a lot more solid.
That's yet another band aid on a hemorrhaging patient. And it still assumes the agent(s) are capable of correct comprehension of the purpose.
1 reply →
The lesson is: write your own code, it's not that hard.
write the tests yourself. Or at least the sketch of the test (to get filled in later). Make a commit of what you do so you can then look at the changes.
If you write the tests the agents have an easier time writing the code!
This reminds me of a discussion a few months back from a BSD maintainer who had done a lot of volunteer work for AWS over the years. I think he might have even been the person who alerted them to the insecurity of IMDSv1. There was a sense that AWS might have had great talented developers at the time, but they clearly didn't really understand the domain of running a hypervisor service exposed to the public. This feels like a similar situation, where I'm sure they "test" what they know to test, in the way they know, but compared to a bank, well, it wouldn't surprise me the least if no one at AWS ever even thought to ask banks how they handle things like this. Instead of testing that a process works the way you specified it, you need to make sure you even have the right specification in the first place by consulting with prior art and ensuring whatever you translate into software reflects the legal reality of the process you're trying to encode.
There's a similar thing with physical processes. My wife works in geointelligence ground processing and encountered this when her system was expanded to support SAR collections instead of just visual spectrum. Software that passed all of its internal tests was passing nonsense collection parameters because the developers didn't understand the difference between energy collected to a sensor cell reflected from the sun versus energy reflected from your own active scan. The capabilities and limitations aren't the same. You can perform the same processing. The bits on disk won't care. But if the process you encode doesn't accurately correspond to the physics of reality, you're producing nonsense. Memory-safe, syntactically-valid nonsense, but nonsense nonetheless.
This seems to happen quite a bit with software companies.
I have bad news if you think banks are the pinnacle of cyber security practices...
Maybe some rare few are, but by and large they generally stink.
They tend to give that impression because they hire tons of auditors and panjandrum, to hassle their suppliers, but internally they are winging it like the rest of us.
The difference is that at bigger banks at least, they tend to throw more money at the problem. They'll have defense in depth with e.g. DMZs, proxies for ingress and egress, locked down workstations and browsers, etc., and a different team for each one of those domains.
So while the actual "on the ground" picture may look suboptimal at any given point, overall it does make for security that in practice is much better than average.
This is reflected in the actual security compromise statistics. Your money in a bank is a lot safer than, say, your credit card deals on file with many retailers.
Does panjandrum mean chaotic rocketry, flahy but worthless in the end?
we invoice clients monthly. if we ever sent one a bill 2^30 times too high, I don't think "estimated charges, no action required" would save the relationship :D
Who needs tests when you have vibes?
It's just like in Superman III
It’s fractions of a penny Peter.
I'm sorry but anyone that sees a multi million or billion dollar bill on an account that does nowhere near that should not be scared. It's obviously a mistake. Stories like this have happened with banks in my country. Check your account and you have billions in there. Guess what happened to those that withdrew money? The judge told them any reasonable person would know this is a bug. Had to give it back. Same thing here, any reasonable person doesn't get scared.
The problem with an anomalous AWS bill is that I don't know if it's because of a bug on their end or because of a goof on my end, eg did I accidentally leave on some giant compute for a month or something. With a billion in my bank account I at least know that I definitely didn't put it in there.
If I experience a bug when transferring money (switching a digit or some such) they’ll shrug their shoulders and say there is nothing they can do. No amount of proof will push them to increment and decrement some integers over there.
They transfer money to me. Their mistake: there is nothing I can do. Rules for me but not for thee?
I’d be concerned as it throws into doubt their entire accounting mechanism
What could a banana possibly cost Michael, ten dollars?
...in the off chance that one in 100,000 will pay their bill instead of dropping dead from a heart attack (perhaps because there's a heartless OpenClaw running their accounting department), it's all worth it.
Why would you think there are "no tests"?
We have a pretty strong existence proof... the thing happened in production. Unless they have some means to override a failing test and scp broken shit to prod, there wasn't a test.
why would a test setting unit to Bytes fail and not MB, KB, or GB, and so on? That's like trying create a unit test for email opt-in, both true and false are valid values. It's up to the user to select the right one.
3 replies →
Technically, there could be a test. It could just be wrong!
3 replies →
missing canaries more likely?
insufficient tests that dont assert on the right things?
the existence of a test doesnt mean it catches the right thing
based on the description, id bet the COE action item will be to do a migration that enforces units are passed at the billing service level
theres no good reason for the billing service to make up its own units.
1 reply →