Comment by lionkor
4 hours ago
LLMs are not another abstraction. ALL OTHER LAYERS you named are fully deterministic, understood, debuggable, etc.
You cannot be serious.
4 hours ago
LLMs are not another abstraction. ALL OTHER LAYERS you named are fully deterministic, understood, debuggable, etc.
You cannot be serious.
A non-deterministic layer seems like exactly what would need a competent, professional to ensure a good outcome, so it doesn't follow that LLM usage would depress wage more than high-level languages depressed wages by opening up programming to tens of millions of people who could never grok assembly.
Counter-point: most developers have no idea or eagerness to actually do that debugging, so it doesn't really matter.
It DOES matter, because the claim that LLMs are a layer of abstraction implies that it's somehow more than a random word generator. It does a great job at generating words in the right order, and often, given enough time, datacenter resources, money, and training, they can produce code that runs and does things as expected.
However, there is absolutely nothing stopping an LLM from "deciding" tomorrow that a fix it built a week ago is no longer real, because not only has that fix left its context, but also the bug was not obvious.
> However, there is absolutely nothing stopping an LLM from "deciding" tomorrow that a fix it built a week ago is no longer real, because not only has that fix left its context, but also the bug was not obvious.
Yeah, and we've never had deterministic tools like GCC suddenly fuck up commonly-relied-on undefined behavior between releases. Sure.
I get what you're saying, but again, to the vast majority of devs, none of that shit matters. Whether that's a good thing or a bad thing is a different discussion.
LLMs are one of the most general abstractions possible.
LLMs are also quite deterministic if you want them to be - generally, their final token selection is deliberately randomized (the model “temperature”). But the word you’re looking for here is probably not actually determinism, it’s probably something closer to predictability.
In any case, it’s perfectly possible to ensure that the output of LLMs is fully deterministic, debuggable, understandable, and testable.
> You cannot be serious.
I don’t think you’re thinking about this clearly.
With a sufficiently complex prompt and a sufficiently complex codebase, LLMs consistently fail and make mistakes, "forget" parts of the prompt, etc.
There's no comparison to be made between this and, for example, a compiler. It's an incompetent comparison.
> I don’t think you’re thinking about this clearly.
My literal job is dealing with layers of abstraction. I'm thinking pretty clearly when I tell you that, not only are LLMs a super leaky, terrible abstraction, they are also not comparable to any other layers of abstraction. All other layers of abstraction we use are well understood, predictable (as you put it), and DEBUGGABLE.
When claude deletes a fix it did two weeks ago, while trying to fix some unrelated error, do you never stop and think "this is not quite the same as what GCC does"?