Comment by jacobsenscott
16 days ago
We've seen this story many times before. Remember UML and Rational Rose, waterfall, big up front design, low/no code services etc. In every case the premise is "pay a few 'big brains' to write the requirements in some 'higher level language' and tools/low skill and low pay engineers can automatically generate the code."
It turns out, to describe a system in enough detail to implement the system, you need to use a programming language. For the system to be performant and secure you need well educated high skill engineers. LLMs aren't going to change that.
Anyway, this is tacitly declaring LLM bankruptcy - the LLM can't understand what to do by reading the most precise specification, the code, so we're going to provide less specific instructions and it will do better?
> It turns out, to describe a system in enough detail to implement the system, you need to use a programming language.
Go back to design patterns. Not the Gang of Four, rather the book where the name and concept was lifted from.
What you will find is that implementations are impacted by factors that are not always intuitive without ancillary information.
It's clear when there is a cowpath through a campus, and the need for a sidewalk becomes apparent. It's not so clear when that happens in code because it often isnt linear. It's why documentation is essential.
"Agile" has made this worse, because the why is often lost, meetings or offline chats lead to tickets with the what and not the why. It's great that those breadcrumbs are linked through commits but the devil is in the details. Even when all the connections exist you often have to chase them through layers of systems and pry them out of people ... emails, old slack messages, paper notes, a photo of a white board.
Oh, I remember. I'm an OpenAPI fan, for example. This time it feels a bit different, though. It won't erase the requirement of having high skill engineers, nor the need to have tech writers create docs, but it might indeed help focus on docs under a different light: not as a post-factum, often neglected artifact, but as a design document. I'm talking of actual user docs here, not PRFAQs or specs.
It always feels a bit different doesn't it. This time it's though. Maybe. No definitely. Will see. We're waiting for AGI anyway, right?
One can only hope. :)
The difference is that LLMs are more capable than the average coder, and rapidly improving. If you give a seasoned developer a loose spec, (s)he'll deal with the ambiguity.
Right, because LLMs cannot learn from their prior work in a project nor can they read a large code base for every prompt. So it can be helpful to provide it documents for understanding the project and also the code base layouts immediately in its context, but you can also generate a lot of that and just fix it up.
the existence of "legacy code" i think is sufficient evidence that the code is not a sufficiently detailed description of a system. if it were, legacy code would not exist, because any engineer could simply read the code and maintain it like the original authors, without needing to say, talk to stake holders or prior documentation, etc.
we're still going to need well educated, skilled engineers, but the LLM context can store both lower fidelity details - eg. "this needs a list, but who cares if its an array or a linked list" and detail not covered by the running code "this looks like it could be a set, but we didnt do that because ..."
hell, semantic web even. we wouldn't need AI