Comment by noosphr

From the good old days: https://152334h.github.io/blog/non-determinism-in-gpt-4/ (that's been a short two years).

Unless something has fundamentally changed since then (which I've not heard about) all sparse models are only deterministic at the batch level, rather than the sample level.