Comment by sublinear
8 hours ago
I agree that documentation should be made higher priority than "coding", but letting AI code everything is throwing the baby out with the bathwater.
It's from this perspective that a lot of engineers feel strong negative sentiment towards AI.
There are always going to be some critical sections of code that one must consider carefully. These tend to be at the extreme ends of choice. Either there's only one way to do it and it probably sucks, or there are many ways to do it and staying optimal is very slippery for maintainers. Identifying and describing these critical sections is also the most important part of the documentation. This is precisely where LLMs fail to do a good job and where people curse the original devs for not writing docs.
But as well, the overall architecture is just as important. The code for this tends to look like the "boring" boilerplate. This is the skeleton of the codebase and LLMs can be bad at this too, haphazardly jamming together design patterns that clash. We're in luck that, usually, a framework or library will provide this code along with documentation to be copypasted verbatim. The rub is when the developers are having to shoehorn it onto existing code they will have to carefully craft some custom interfaces and document them very well.
So in the end, what's left for the LLM to do? When it does a good job, it's usually cribbing so heavily from existing solutions by humans you could have copied it yourself if you knew where it got it from. The LLM is automating copypasta, not deliberate coding. When it's bad, it's making a mess only suitable for a rough proof of concept, if it even works.
From the perspective of a diligent engineer trying to avoid technical debt and other incidents down the road, burning an extra couple of days to get it done right by hand isn't that big of a deal. LLMs become about as useful as a google search. Assuming one does not work at a coding sweatshop, why not just use the google AI summary 90% of the time? The agentic workflow doesn't look promising for a significant chunk of experienced engineers working on maintenance more often than new projects.
> When it does a good job, it's usually cribbing so heavily from existing solutions by humans you could have copied it yourself if you knew where it got it from. The LLM is automating copypasta, not deliberate coding
I've been saying this a lot lately. I agree completely. Most of the impressive stuff I see people talking about AI, they could have just copied and pasted from open source repos in the past
I think that was sort of a taboo in the past though, for several reasons. First off, most teams I've worked on were reluctant to take on the additional maintenance of unvetted open source code if there were other options. For some reason this reluctance has flown completely out the window with AI, even though the cost of code maintenance hasn't really decreased.
Secondly, there was always the question of code licenses when copying from an open source repo. Companies were reluctant to risk anything that could be viewed as code theft
Now the code theft part is solved by training LLMs instead of just copying and pasting, which just feels like theft with extra steps to me
> From the perspective of a diligent engineer trying to avoid technical debt and other incidents down the road, burning an extra couple of days to get it done right by hand isn't that big of a deal. LLMs become about as useful as a google search. Assuming one does not work at a coding sweatshop, why not just use the google AI summary 90% of the time?
If there's one thing that seems unclear from AI proponents, that is the number of technology they're using at the same time. After months or so on a project there should be enough that should be in longterm memory or resources that are part of your browser history that things that require deep research should be a small part of each ticket. And it would be important enough to get it right.