Comment by jknoepfler
1 day ago
> "superficially superior code"
Can you unpack that a bit? It produces measurably, meaningfully inferior code everywhere I see it in use.
> "leadership encourages this because from what they can tell, there is no downside"
As said 'leadership' I find this a bit puzzling. I'm seeing strong, quantifiable evidence of increasing churn, increasing incident count, and length of downtime from the date of our biggest push into GenAI, and I'm organizing efforts on my teams to mitigate those issues and actively reduce GenAI adoption.
If you mean my c-suite, you're mostly correct although they are already rumbling about seeing zero or negative ROI on GenAI investments.
> Anyone not using LLMs all day is just not going to be as prolific
Agreed, but prolific != productive.
Sure, by "superficially superior code" I mean that LLMs, by design, produce code that has syntax and idioms that are perfectly in line with industry norms due to the vast training corpus that it comes from. It looks great at first glance.
Normally, this is a signal that the developer is at least informed about best practices in the language and is usually a pretty good indicator that they're thinking clearly about the work they're doing. That, along with reputation, takes a lot of cognitive burden off the reviewer.
However, with LLMs that is all it means. You can be sure the code is norm-compliant, but in my experience that is all you can be sure of. There's not really a guarantee that it has thought deeply about the problem space.
It's like a bridge that looks great.
> leadership
Not sure if its just me, lately I have started feeling pretty offensive about the increased usage of the word. Its management not leadership by any means.
> "leadership encourages this because from what they can tell, there is no downside"
For most people in management its easier to pick the current set of slangs/abbreviation's, general trend and go with it. Understanding the details would take time, raise questions and no one in management has time or political capital to spend on it.
What size company are you talking about?
The code written by agents generally seems to reflect what you (the human driver) asked for. If you discuss the approach and architecture first, asking the right questions in the process, and then let it implement - the result is quite on point with the frontier models. Might need some minor touch-ups if agent missed some common conventions or guessed the expectations wrong (but again, salvageable with follow-up prompts). And when it comes to line-by-line logic within functions, I would argue that today's models are LESS likely to make a mistake in there than humans - especially if you cross-review with another model (e.g. "write with Claude, review with GPT").
Humans write slop too, you know. Just saying.