Comment by devin
3 days ago
Some conjecture on my part, but I imagine that the bugs being found are not the kind you can just press a button on the LLM to fix anyway, but are instead the sort of bugs which require the reviewer to deeply consider design tradeoffs, of which I believe there are simply more in a DB or programming language codebase, for example. Stuff I am finding in review is often like "this thing isn't like the other thing, which has this behavior, which is wrong if the aim is X, but sort of correct if the aim is Y, and totally perfect is the goal is Z and the use case is J, but not I. Unless it's Q, in which case you probably need to reconsider some fundamental decision making from 12 months ago and have some conversations with people about how users desire it to work in the first place, and then set about understanding the tradeoffs inherent to that design."
ETA: This is extremely generous to the LLMs, btw. If I got an LLM to pop something this concise out I would fully commit to finding a different profession tomorrow, but for now I am for better or worse stuck doing prompting some non-zero number of times in order to get all of this assembled context, and then need to use my human powers to decide whether or not more humans indeed need to discuss it.
I like the way you put this. I agree this is where the frontier of software development is right now. You can have LLM chug and bug fix all day long but in the end most real issues are in this layer of intent. I don't think this problem is particular to DB/programming language codebase either - it happens a lot in product design it just doesn't surface in code review. "Everything is working perfectly" yet the resulting workflow or UI is not truly making the user's life easier, or has grown grotesque with additions/modifications.
I will say, if the problem is purely code you occasionally see "sparks of AGI" where the model will step back and say "hold on, there is a massive simplification we can make here...". This started happening with Fable-class models. But rarely if ever will the model push for a huge simplification at the product level.
In my experience it's the opposite - the model's design decisions always try to add complexity, even when it's not necessary.
So you get permission flags and multilayered data structured with complex interfaces, and there is never any sense of "Wait a minute - we don't need this."
It's much like the way AI art looks meaninglessly busy with too many random lines or other elements, and AI writing is always exactly, precisely, specifically, unnecessarily adverbial.
>> In my experience it's the opposite - the model's design decisions always try to add complexity, even when it's not necessary.
This happens if the model does not have good context regarding what is being built, why and for whom.
For one of my side projects, my CLAUDE.md has a line that says "the users of this product will be non-technical people who find technology scary." And you know what? Claude always steers me away from complexity when working on that project.
1 reply →
Yes > real issues are in this layer of intent
This is why code is going to be automated, and the software engineer is a link between organizational intent and the code. This is what I’m working on.
How does this differ from what I'm doing today? Want to say any more on the topic? This is a bit vague.
5 replies →
You nailed it! I often got bug reports from AI that would touch far more code to fix than what I am willing to touch, and the likelihood of that large piece of new code (which I honestly would not be able to review thoroughly) will likely contain more bugs than the one I am fixing.
So in the end I often ignore these bug reports after evaluating their actual impact on the product.
And imagine you had an LLM ingest the entire corpus of comments and discussion threads for all of Postgres' history, and asked it to make a judgment call on the tradeoffs - or even on how to present the tradeoffs for discussion.
How much of its judgment would be driven by the data vs. its own internal knowledge? How much might be driven by old vs. new discussions that might not represent new directions of the project? How much might be driven by over-matching against how bugs in less mission-critical code paths were handled?
There's no way to quantify the answers to these questions. Which is not to say this applies to every problem out there! But human judgment is still a bottleneck for design questions on widely, disparately-used complex systems.
ETA: This is extremely generous to the LLMs, btw. If I got an LLM to pop something this concise out I would fully commit to finding a different profession tomorrow
Many of us have already experienced that uncomfortable revelation, where a leading-edge LLM actually comments correctly on our questionable judgement. You will, too, soon enough.
I don’t think you followed the point being made.
If I got an LLM to pop something this concise out I would fully commit to finding a different profession tomorrow
Help me out. What'd I miss?
In my case, rather than finding a different profession, I chose to walk through the open door to the next level of the same profession.
2 replies →