Comment by cortesoft
8 hours ago
No, LLMs ARE deterministic, just like all computer programs are.
I get why that is in practice different then the manner in which compilers are deterministic, but my point is the difference isnt because of determinism.
I think you are misunderstanding the term "deterministic". Running on deterministic hardware does not mean an algorithm is deterministic.
Create a program that reads from /dev/random (not urandom). It's not determistic.
Fair, although you can absolutely use local LLMs in a deterministic way (by using fixed seeds for the random number generation), and my point is that even if you did that with your LLM, it wouldn't change the feeling someone has about not being able to reason out what was happening.
In other words, it isn't the random number part of LLMs that make them seem like a black box and unpredictable, but rather the complexity of the underlying model. Even if you ran it in a deterministic way, I don't think people would suddenly feel more confident about the outputted code.