Comment by gste
2 hours ago
I think this is outdated. If you follow spec-driven development, get the model to do all the planning work upfront, review and iterate the plan, write clear markdown file documentation on the abstractions and patterns you want to follow, then you have every opportunity to tell the model how you want it to write the code. If you use Opus or Fable 5 it will then write the code better and faster than you will.
you're a lot more skilled than I am if you're able to know the abstractions and patterns you want while only have a weak grasp on the actual code. For me the abstractions emerge as I understand what I'm actually working with
This just doesn't match my experience writing idiomatic C code for embedded real-time systems with ChatGPT-5.5 and -5.6. Are Opus and Fable really that different? I general I find that a fully correct implementation for some function may take an hour and be nearly one-shot, but going from there to code I'd be willing to actually commit is at least 4x the time commitment with regular back-and-forth -- so four hours of deep engagement for every one hour of one-shot. ChatGPT-5.6 Sol seems to be better that -5.5 at the one hour do-it portion, but worse at the four hour do-it-well portion.
IMO "get the model to do all the planning work upfront, review and iterate the plan" is backwards. It works better if you do an initial plan yourself and then have the AI review it (and iterate as necessary). Makes you think about the design for a bit so you can have a semblance of a mental model.
I have never seen a plan written by LLM that wasn’t vague and light on details, every time I need to ask for more details and every time I ask to implement it trips over some dead end in the plan, discards the plan and continues as if there was no plan to begin with. Planning feels often just narrating the request and a wish list then what a human would do - methodically build enough understanding so that you’re confident of the direction you choose. LLM plans are overhyped and overrated.
Sure. But try it on a project on non-trivial size and then tell us the cost of it. LLMS are billed per token.
Depends on how long that planning/review cycle takes.
Agreed, all of the staff/principal engineers I work with are ~99%+ AI generated code, and increasing business value delivered as a result. This is on planet scale infra not CRUD apps. (And yes, you do need to carefully review the output and give steers/corrections. It’s still faster.)
At this point if you can’t get the agent to write good code then either I) you are in a very specific niche (like Karpathy trying to write NanoGPT) that is extremely out-of-distribution, or II) skill issue, you need to learn how to prompt better.
It’s fine to have a skill gap! Just don’t delude yourself that the tools are bad and everyone claiming they are good is wrong.
How is a GPT implementation “extremely out of distribution” but “planet scale infra” isn’t? That’s got me totally confused about your point that I was taking seriously.
A lot of stuff that underpins planet-scale infra is open source and has been for a long time. The general approaches and architectures for it have also been widely discussed and litigated in the commons so it's not just the code - the specific why is also very well documented.
So despite its importance much of it is actually pretty in-distribution.
1 reply →
it's really cool how in the past measuring developer productivity was incredibly difficult, but now its become easy if someone's making a point about LLMs
btw, your last paragraph makes you sound like an ass.