Comment by adw

> But is this "stochastic nature" inherent to the LLM?

At any kind of reasonable scale, yes. CUDA accelerators, like most distributed systems, are nondeterministic, even at zero temperature (which you don't want) with fixed seed.