Comment by cortesoft
12 hours ago
LLMs are deterministic, too. I know there is randomness in the choosing tokens, but that randomness is derived from a random seed that can be repeated.
12 hours ago
LLMs are deterministic, too. I know there is randomness in the choosing tokens, but that randomness is derived from a random seed that can be repeated.
Only if the seed is known. Determinism is often predicated on perfect information. Many programs do not have that. Their operations cannot be reproduced practically. The difference between saying deterministic and non-deterministic is contextual based on if you are concerned with theory or practicality.
If I understand your argument, you're saying that models can be deterministic, right?
Care to point to any that are set up to be deterministic?
Did you ever stop to think about why no one can get any use out of a model with temp set to zero?
llama.cpp is deterministic when run with a specified PRNG seed, at least when running on CPU without caching. This is true regardless of temperature. But when people say "non-deterministic", they really mean something closer to "chaotic", i.e. the output can vary greatly with small changes to input, and there is no reliable way to predict when this will happen without running the full calculation. This is very different behavior from traditional compilers.
No, LLMs ARE deterministic, just like all computer programs are.
I get why that is in practice different then the manner in which compilers are deterministic, but my point is the difference isnt because of determinism.
I think you are misunderstanding the term "deterministic". Running on deterministic hardware does not mean an algorithm is deterministic.
Create a program that reads from /dev/random (not urandom). It's not determistic.
1 reply →