Comment by yuchen20

When I was at AWS, they famously required an extensive "CoE", correction of errors, or post-mortem, in an instance of over-charging a customer $0.26.

The idea is that if we can make small billing mistakes like that, we can make large billing mistakes, and need to invest in the correctness of the systems powering billing.

I have great respect for the engineering culture within AWS during those times. I am glad to have left before seeing it degrade and decline.

The irony is, the only purely deterministic thing, will be token consumption...

I’m not so sure about that. I can see a real rationale for creating sanity checks using AI to more quickly/proactively catch pathological billing issues before they become HN nightmare stories. They wouldn’t replace billing code, but there are many ways that stupid customer mistakes can cause real costs to Amazon that either have to be refunded and absorbed by Amazon or paid by the customer causing a negative opinion of AWS. If a billing AI watching costs in realtime could detect, say, a lambda loop in the first 10 min and either alert the customer or kill it, that would make AWS feel a lot safer to use. Enumerating these conditions and fixing them individually is a task that Amazon has proven incapable of achieving. An AI watchdog layer might be the perfect shortcut to addressing all of these problems at once. Because it’s well-trodden territory that AWS has so many multi-thousand dollar foot guns that make it really scary to use as a hobbyist or small business on a tight budget.

It doesn’t matter which domain it is, AI should be utilized at anomaly / patterns detection and prototyping, with goal of a prototype to become coded as discovered pathway.

I don’t understand why companies still place AI in their system at authority-forming level, if anything it should sit at least behind single guard / validator that is explicitly coded and tested, never in front of presentation layer.

This is like half of all job listings I've read recently. And it's a decent amount of fintech that's like this.

Probably not actually. Transferring one kilobyte across a network link has such a low value that the billing costs of aggregating it cost more than the revenue.

So instead you take a probabilistic approach - charge the user for a megabyte of data transfer 0.1% of the time, and bill nothing 99.9% of the time.

Now the typical cost is the same, the users bill is probably accurate to the cent, but you have divided the number of billing records by 1000.

Just wait until the same system runs payroll and you're getting paid $1.94400 annually.

That's base, don't forget bonus and stock.

It gets worst:

"Senior Software Development Manager, AWS Global Bill Generation" https://www.amazon.jobs/de/jobs/10471948/senior-software-dev...

"We're transforming from monthly batch processing and manual war rooms to continuous billing, autonomous agents, and self-healing infrastructure. We believe operational burden is a technical problem, not a staffing problem"

This looks clearly...a staffing problem...

Seriously! If I were making a joke I would say something like

> Build AI-driven continuous validation frameworks powered by agentic workflows and large language models that autonomously manage…

But that’s word for word a 250k+ TC job in the big ‘26.

> enabling domain experts to review in hours what previously took weeks.

This is a gold-mine. They need to get sued heavily for this incompetence.

It's true, if you're spending that much money you must be worth a ton! Just look at SpaceX!

Big numbers can lead to stress which can lead to all kinds of disorders.

If you were a business maybe you could claim for the emergency on-call time spent diagnosing, but you'd probably still lose AND amazon would fire you as a customer.