Comment by verdverm
3 months ago
It occurs to me this pattern might be the average code we humans have produced. We all have made those quick fixes, copy-pastas, and dirty hacks... they learned it somewhere! I also assume that some of the behavior is an artifact of their training regime.
So with LLM outputting average code, and people using LLM more and more, I guess the average code will become worse over time ?
There is a belief that everyone is just taking whatever the LLM (really agents now) outputs. This is not the case anywhere I work. We use human oversight to have it iteratively improve the code. The average quality is going up.
Do you have some metrics to back this up ? Because from what I can see with my own eyes, outages everywhere, security holes everywhere too, doesn't seem that things are improving..
1 reply →
Not advocating for AI code slop--but if AI coded software works correctly, maybe it doesn't matter? Except sometimes when a specialist will have to get involved. Not a perfect analogy, but most people don't write assembly these days--they have a compiler do that. Assembly still has a place, but it's a specialist task.
> if AI coded software works correctly, maybe it doesn't matter?
The problem isn't the amount of code, it's how fitting/unfitting the abstractions are. Wrong abstractions are bugs in waiting. If there's much code with wrong abstractions, future change becomes difficult.
Source: me, I've created many bad abstractions and they led to much pain...
3 replies →
More verbose code takes up more space in the context. It's harder for humans to review, but also harder for future AIs to edit. Unless you manage to keep the AI to firm module boundaries & have it replace modules wholesale it's not really equivalent to how assembly gets replaced wholesale when a compilation unit changes. Compilers aren't editing the `.o` files when you rebuild, they throw the old ones out & replace them. But when you prompt an AI it is reading & editing the source files, so excess verbosity in the source files is detrimental.
Well, if tokens = cost, and verbosity = more tokens, then smaller code is a financial (and human!) win. Although I'm worried vibe coders are just going to have LLMs modify minified code in caveman mode so they can have 100 agents in a swarm..
On a more serious note, I wonder if this might eventually encourage people to use languages that are a little harder to write but much more concise (functional languages for instance). When you're paying per-token enterprise bean java style verbosity totally sucks
But the truth is: it doesn't work correctly. I see quality of software dropped significantly.
At work we are integrating with third party platform to automate excel-powered calculations. It is awful. Rendering the table in browser takes 10s or one click on Export button will throw backend in OutOfMemory state.
1 reply →
In my case, where I see it most often is when the LLM has to rework something multiple times, and the feedback loop is vague (especially when all I have to give it is "no error messages, but it's still broken"). It seems like after the third or fourth try it just kinda goes off the rails. I find that the one-shot quality tends to be a little better, if the slot machine happened to work correctly that time.
You shouldn't be using an LLM directly (web chat style). A proper harness allows an agent to see the errors itself and correct as needed. You can the correct it at higher, more meaningful levels.
My experience is in doing this with Claude/Codex/OpenCode with a pretty rigorous setup (AGENTS.md/CLAUDE.md for specific subfolder rules, strict compile/test/lint rules. This isn't me copy-pasting from web chat.
1 reply →