Comment by jamesmcq
4 days ago
The key part of my comment is "correctly".
Does it write maintainable code? Does it write extensible code? Does it write secure code? Does it write performant code?
My experience has been it failing most of these. The code might "work", but it's not good for anything more than trivial, well defined functions (that probably appeared in it's training data written by humans). LLMs have a fundamental lack of understanding of what they're doing, and it's obvious when you look at the finer points of the outcomes.
That said, I'm sure you could write detailed enough specs and provide enough examples to resolve these issues, but that's the point of my original comment - if you're just writing specs instead of code you're not gaining anything.
I find “maintainable code” the hardest bias to let go of. 15+ years of coding and design patterns are hard to let go.
But the aha moment for me was what’s maintainable by AI vs by me by hand are on different realms. So maintainable has to evolve from good human design patterns to good AI patterns.
Specs are worth it IMO. Not because if I can spec, I could’ve coded anyway. But because I gain all the insight and capabilities of AI, while minimizing the gotchas and edge failures.
> But the aha moment for me was what’s maintainable by AI vs by me by hand are on different realms. So maintainable has to evolve from good human design patterns to good AI patterns.
How do you square that with the idea that all the code still has to be reviewed by humans? Yourself, and your coworkers
I picture like semi conductors; the 5nm process is so absurdly complex that operators can't just peek into the system easily. I imagine I'm just so used to hand crafting code that I can't imagine not being able to peek in.
So maybe it's that we won't be reviewing by hand anymore? I.e. it's LLMs all the way down. Trying to embrace that style of development lately as unnatural as it feels. We're obv not 100% there yet but Claude Opus is a significant step in that direction and they keep getting better and better.
3 replies →
> But the aha moment for me was what’s maintainable by AI vs by me by hand are on different realms
I don't find that LLMs are any more likely than humans to remember to update all of the places it wrote redundant functions. Generally far less likely, actually. So forgive me for treating this claim with a massive grain of salt.
Yes to all of these.
Here's the rub, I can spin up multiple agents in separate shells. One is prompted to build out <feature>, following the pattern the author/OP described. Another is prompted to review the plan/changes and keep an eye out for specific things (code smells, non-scalable architecture, duplicated code, etc. etc.). And then another agent is going to get fed that review and do their own analysis. Pass that back to the original agent once it finishes.
Less time, cleaner code, and the REALLY awesome thing is that I can do this across multiple features at the same time, even across different codebases or applications.
To answer all of your questions:
yes, if I steer it properly.
It's very good at spotting design patterns, and implementing them. It doesn't always know where or how to implement them, but that's my job.
The specs and syntactic sugar are just nice quality of life benefits.
You’d be building blocks which compound over time. That’s been my experience anyway.
The compounding is much greater than my brain can do on its own.