Comment by joshheitzman
4 hours ago
People, especially those who don't do software development, often conflate coding and software development.
The models aren't good at architecture and design. But they take direction on architecture and design and design well and can refactor code quite effectively. AI agents can absolutely be used to clean up vibe coded code bases once you figure out if the investment is worth it. The mess can be avoided if you give them sufficient guidance on architecture and design upfront.
That said, doing so purely in text form doesn't feel great right now. I've been thinking about UML lately. The problem with that was the roundtrip after the code was generated and then the implenetation happened. I don't necessarily think UML is the solution, but neither is walls of dense text.
>The models aren't good at architecture and design.
Can you elaborate to back up this claim? WHat exactly is your yardstick for "being good at SW design and architecture"?
Because I found the current SOTA AI models being great at architecture and design, much better in fact than most average real-world devs. Is your yardstick just the John Carmacks of the world by any chance? Because most devs are not John Carmack. They are also not Linus Torvalds, they are not Stallmann, etc.
Maybe your LLM experience is still stuck in the 2023 era of ChatGPT?
My yardstick is me: https://www.linkedin.com/in/joshheitzman
Cool. Can you elaborate at which types of task you are better than SOTA LLMs in context of "being good at SW design and architecture"? Got any examples? Is it at the interview questions? Or real world problems? If so what is the scale of the real world SW design and architecture problems you're better than the SOTA LLMs? Is it FAANG scale or mom and pop shop scale?
And do you consider yourself to be representative of the average developer, above them, or below them?
LLMs don't even need to be better than the average dev, let alone the top performing ones, like you. If they can just be better than the bottom 20% of devs and white collar workers in general(easily achievable when you've been around the block and saw how many useless people just keep warm chairs for high wages in large companies), that's already a huge win for those products.
What I mean, at a previous job I had ran into a memory leak issue in our backend and discovered a colleague pushed a library into prod which came with comments in the source code saying "DO NOT USE IN PROD, IT CAUSES A MEMORY LEAK!". There's cases where human stupidity and carelessness far surpasses whatever issues LLMs cause so maybe the average dev isn't really that much better than the SOTA LLMs.
1 reply →