Comment by lelanthran
2 days ago
> I feel like using a compiler is in a sense a code generator where you don't commit the actual output
Compilers are deterministic. Given the same input you always get the same output so there's no reason to store the output. If you don't get the same output we call it a compiler bug!
LLMs do not work this way.
(Aside: Am I the only one who feels that the entire AI industry is predicated on replacing only development positions? we're looking at, what, 100bn invested, with almost no reduce in customer's operating costs other than if the customer has developers).
> Compilers are deterministic. Given the same input you always get the same output
Except when they aren't. See for instance https://gcc.gnu.org/onlinedocs/gcc-15.1.0/gcc/Developer-Opti... or the __DATE__/__TIME__ macros.
From the link:
> You can use the -frandom-seed option to produce reproducibly identical object files.
Deterministic.
Also, with regard to __DATE__/__TIME__ macros, those are deterministic, because the current date and time are part of the inputs.
Determinism is predicated on what you consider to be the relevant inputs.
Many compilers are not deterministic when only considering the source files or even the current time. For example, any output produced by iterating over a hash table with pointer keys is likely to depend on ASLR and thus be nondetermistic unless you consider the ASLR randomization to be one of the inputs. Any output that depends on directory iteration order is likely to be consistent on a single computer but vary across computers.
LLMs aren’t magic. They’re software running on inputs like anything else, which means they’re deterministic if you constrain all the inputs.
4 replies →
Why does it matter to you if the code generator is deterministic? The code is.
If LLM generation was like a Makefile step, part of your build process, this concern would make a lot of sense. But nobody, anywhere, does that.
> If LLM generation was like a Makefile step, part of your build process, this concern would make a lot of sense. But nobody, anywhere, does that.
Top level comment of this thread, quoting the article:
> Reading through these commits sparked an idea: what if we treated prompts as the actual source code? Imagine version control systems where you commit the prompts used to generate features rather than the resulting implementation.
Ohhhhhhh. Thanks for clearing this up for me. I felt like I was going a little crazy (because, having missed that part of the thread, I sort of was). Appreciated!
LLMs CAN be deterministic. You can control the temperature to get the same output repeatedly.
Although I don’t really understand why you’d only want to store prompts…
What if that model is no longer available?
They’re typically not, since they typically rely on operators that aren’t (e.g. atomics).