Comment by hansvm
4 days ago
Yeah, it's not the non-determinism; it's that they don't do what I say. I can't imagine writing a nontrivial program without, at a minimum, reading and editing the pieces as they're being produced. People aren't complaining about non-determinism; they're complaining about the results being non-deterministically sometimes wrong.
And if they were deterministically sometimes wrong, that wouldn't be better.
That actually would be significantly better, because then you could work out under what conditions they were wrong and correct for it. But being stochastically wrong means you have to check everything, because you never know when the errors will creep in.
If it were deterministic but chaotic, which AFAIK is the case for LLMs running at temperature 0, then you would have to likewise check every time you changed anything and couldn't meaningfully correct for it.
If the results are non-deterministic, unless I'm happy with literally any possible result, then yeah, I'm going to be unhappy with all of the other results. That's what non-determinism means. What would being right always with non-deterministic results even look like in terms of code output?
Compilers have a ton of hard optimization passes, and many of those are solved well enough via stochastic algorithms. Nobody actually fully optimizes register allocation or whatever, and a few random restarts can go a long way toward some better performance.
Every professional compiler I've ever used has given me the ability to choose what optimization level to use. No professional LLM I've ever used has given me any choice whatsoever to modify the temperature.
This argument is conflating determinism as in "byte-for-byte reproducible builds" with determinism as in "builds are semantically correct".
The fact that a traditional PL compiler output can have different performance, based on stochastic optimizations, is not in dispute. But, we consider the compiler flawed and inappropriate for use when it will randomly produce incorrect machine code. Under normal circumstances, we can infer that "correct" source code will produce "correct" builds every time it is fed through a valid compiler.
The problem people have with slop and vibe coding is that there is no formal semantics to even define correctness of the LLM/agent output.
With no formal semantics to constrain the outputs, you have to treat each prompt/inference run as a fuzzy search. What it "finds" may or may not match your intended semantics. It is merely a candidate that might correspond to your ambiguously specified semantic intent.
It is magical and lazy thinking that leads people to use this fuzzy search process in "I'm feeling lucky" mode.
No compiler does optimal register allocation, because optimal register allocation is an NP-complete problem.
Chaitin et al.: "Register allocation via coloring".
https://en.wikipedia.org/wiki/Register_allocation
1 reply →
How often do programmers do what managers say, and how often do managers say what is wanted or needed?
It seems like your points are:
1. Programmers sometimes do something other than what they're told, so what's the harm in letting an LLM be disobedient instead?
2. Managers sometimes don't ask for the right things, so why not let an LLM figure things out?
To both points, those observations are literally the reason why we have software engineers writing code instead of management. The job of fully understanding the implications of all requirement interactions in excruciating detail is not a management responsibility. In the context of using LLMs as a higher-level compiler, I don't think people are arguing that management should be the people writing that higher-level code.
Separately, the failure modes are very different. If I ask a SOTA agent and a SOTA harness to construct a minimal sandbox for executing subprocesses without network access using OS sandboxing primitives to ensure you don't have Scun*horpe-esque ways of sneaking networked commands through (and assume I expanded that into a proper prompt instead of the high-level summary I'm using to convey the idea here), there's a nontrivial chance it implements some of the sandbox by string matching the commands instead of using proper OS primitives. It'll have thousands of tests showing it works, and it'll still be garbage even just from a requirements point of view and ignoring code quality or everything else we're being told to ignore when using them as higher-level compilers. There's also a nontrivial chance that if you ask either the same or a new session to check on that property it will conclude that the sandbox was implemented correctly. Good human engineers will either get the job done or tell you that they can't for some reason or another. That "looks like it succeeded but actually failed hard and you can't tell unless you read the code" error case is new to LLMs. Yeah, we write bugs from time to time (as do LLMs), but complete failures to understand the core product so badly that fixing the "bug" requires a full rewrite, one likely to introduce its own similarly catastrophic rewrite-inducing bugs, are a different beast altogether.
And again, I use LLMs a lot, almost exclusively, but they have limitations, and failing to recognize those isn't conducive to success. The frontier keeps moving, so you might find these current observations irrelevant eventually. People are choosing to not read the code today though, and that's bad.
>In the context of using LLMs as a higher-level compiler, I don't think people are arguing that management should be the people writing that higher-level code.
I felt this was somewhat arguing it https://news.ycombinator.com/item?id=50114047
>It seems like your points are:
>1. Programmers sometimes do something other than what they're told, so what's the harm in letting an LLM be disobedient instead?
1. People spend a lot of time talking about how LLMs often do something other than what they are told, as if people forget that they aren't especially good at doing what they're told either.