Comment by Retric
16 days ago
Optimization level, target architecture, etc are just information fed to the compiler. If it’s still nondeterministic with everything kept the same your compiler is broken.
16 days ago
Optimization level, target architecture, etc are just information fed to the compiler. If it’s still nondeterministic with everything kept the same your compiler is broken.
You're missing the point. I'm not saying that compilers are nondeterminitsic or that LLMs are deterministic. I'm saying that it doesn't matter. No one cares about deterministic results except programmers. The non-technical user who will make software in the future just knows that he gets what he asked for.
Systems randomly failing is of significant concern to non programmers, that’s inherent to the non-deterministic nature of LLM’s.
I can send specific LLM output to QA, I can’t ask QA to validate that this prompt will always produce bug free code even for future versions of the AI.
Huh? No.
The output of the LLM is nondeterministic, meaning that the same input to the LLM will result in different output from the LLM.
That has nothing to do with weather the code itself is deterministic. If the LLM produces non-deterministic code, that's a bug, which hopefully will be caught by another sub-agent before production. But there's no reason to assume that programs created by LLMs are non-deterministic just because the LLMs themselves are. After all, humans are non-deterministic.
> I can send specific LLM output to QA, I can’t ask QA to validate that this prompt will always produce bug free code even for future versions of the AI.
This is a crazy scenario that does not correspond to how anyone uses LLMs.
1 reply →