Comment by drob518
12 hours ago
My Spidey sense was tingling when I saw that, too. An additional issue is how humans are supposed to read the code at all so that they can provide help to the LLM if it’s off track. If the code is only usable by models, the models need to be good enough to deal with binary feedback (“Code doesn’t work.”). The human won’t be able to read the code and steer the model. Given the levels of steering required today, that makes me quite nervous.
Even if the target author is an LLM, the accountability still lands on humans eventually
I guess the point is that there is no need for humans to read the code.
How often do you read assembly to check what your compiler is doing?
There is a niche of people doing it when they have special constraints, but that's a tiny niche.
> How often do you read assembly to check what your compiler is doing?
The difference is my compiler is more-or-less deterministic, and tends to do exactly what the specification provided to it (the source code) says. LLMs do not currently fulfil either of those criteria