Comment by zelphirkalt
2 days ago
I would like to state here, that LLMs are actually entirely deterministic. The temperature is just as much of an input as any other parameter and instructions and prompts. Deterministic does not mean "explainable" or "reasonable" or "understandable". Unless LLMs somehow have broken out of the confines of our very deterministic computers/calculation machines without us noticing, they are very much as deterministic as anything else we run on these machines.
What people perhaps mean by non-deterministic on a computer is, that we often do not control all inputs and therefore cannot tell what the output will be. That's however actually more about our own inability, rather than the processes within the machines.
If the temperature dependency is implemented via a deterministic pseudo-random algorithm, then they are deterministic. If, on the other hand, the output is at least partially causally dependent on a source of environmental randomness, then, even though everything 'inside the box' is deterministic, the process as a whole may or may not be, depending on whether the source of environmental randomness is, and that is ultimately a question about fundamental reality.