Comment by azuanrb

Eventually infrastructure will be more simple to orchestrate too without faults I suspect from well developed devops harnesses. The risk and scale companies are willing to accept will still fall on humans for some time even then. I don't see most people vibe coding a million user app that has deeper needs than the basics we see now.

Can you elaborate more on this type of role? Stack? Etc.

Long codex sessions lead to a lot of cached token hits, esp when you resume them after a few hours.

Vibe coded a simple game (10,000 tokens of source code) with two popular coding agents. (Once each, to compare.)

One spent 200,000 tokens, to produce 10,000.

The other spent 1.9 million.

It could have been a single LLM call (10k tokens). lmao

(I note that the latter was designed by a company whose main source of revenue is token spend...)

Don’t forget context. Basically I have 2 billion input and 1 million output. Every prompt you do, sends back the whole thing again and again. Let’s say you have 500k context used, you send 10 messages is 5 million. 100 messages 50 million. Use 5 threats is 250 million.

[dead]

I agree that a consistent QA mindset is rare, but I'm not sure even if present if it's enough to replace an SDE.

I very recently looked at the codebase of a vibe-coded app made by someone with domain expertise but no software dev experience.

It was very clear to me that he had described it from his POV to an AI, and the AI had implemented features in a manner that technically worked, but made future maintenance or expansion extremely tricky, which is why he was now looking for a dev.

For example, in his data schema, for every item on a menu, instead of simply having an array property like so for ingredients:

    items["latte"]["ingredients"] = ["water", "milk", "sugar", ...]

He had individual flags for every item for every possible ingredient it could have or not have:

    items["latte"]["has_milk"] = true
    items["latte"]["has_nutmeg"] = false
    items["latte"]["has_cinnamon"] = false
    items["latte"]["has_sugar"] = true
    ...

This technically worked and passed tests from his POV at an MVP level. But added a lot of complications when actually trying to build more features or when a new menu item had ingredients the founder hadn't thought to include in the schema beforehand.

I totally get how he ended up where he did though. While describing it to the AI, he probably said something like "store info on each menu item's ingredients, they might have milk or coffee or sugar", and the AI created individual flags for them and he didn't think to question it, because he didn't know what's "right" or "wrong", but then as he kept building the AI stuck with keeping individual flags instead of swapping it out with an array mechanism, and he couldn't have known the correct way to implement it.

Only a dev with experience would know how to describe the system to an AI model to get an output that works well, and how to assess the quality of its output beyond what can be assessed through the basic UI. This wasn't a QA failure, it was a design failure.

I disagree. At some point of complexity, building it yourself is faster, better and (as we're finding out) cheaper. And more fun, although that varies person to person.

Wrestling with a code generator also creates a sunk cost fallacy where progress grinds to a halt but you still try and use the tools to fix the problems the tools created. Or you go in and fix things yourself, in a codebase you don't truly understand. A single developer can recreate the contextual nightmare miasma of a large corporation all by themselves.

There's also an emerging market consideration: MVP are easy to build so time to market is no longer hard to achieve. It's not a differentiator.

X was built in 3 days but is slow and riddled with bugs and security errors. There are also A, B, C, D and E which are effectively the same thing built just as fast.

Z was built over six months and is rock solid and performant.

Who wins the market share?

Personally my ability to understand atrophies / is reduced when compared to writing code ‘myself’ rather than fully being a reviewer.

Probably similar to hand writing notes (while digesting + synthesizing and not just being a scribe) vs reading notes somebody else took.

> Domain expertise combined with a QA mindset could replace SWE, but consistent QA mindset is rare

I've heard this story at least 3 times already:

- Domain expertise combined with outsource could replace expensive US SWE

- Domain expertise combined with SWE could replace QA

- Domain expertise combined with SWE could replace infra engineers

Why is everyone so preoccupied with replacing someone with someone instead of doing their fucking job?

You can't test quality into a product. Regardless of how much of a "QA mindset" you have, you can only ever find a fraction of defects and technical debt through external testing. This can be good enough for a throwaway app that will only be used by a limited customer base for a limited time. But that approach quickly bogs down if you try to scale it into a product that will be used indefinitely by a huge set of external customers. At some point velocity drops to near zero because the code base is such a mess that it becomes impossible to add new features without causing regression defects or breaking backward compatibility.

The engineering part of software engineering is the hard part for LLMs. How is that replaceable with these skills?

I don’t think so. Most things are sufficiently complicated enough to require multiple domain experts working together to achieve a goal.

The dunning kruger effect is in full swing as people think AI replaces the domain expert need.

Most of the value in the expert isnt the 80% but the tail 20% or 10% where AI fails. For a one of personal app or website, 80% is plenty but only that.