Comment by Iolaum
13 hours ago
Was the temperature 0? Cause unless I don't understand it right, any non-zero temperature implies probabilistic next token prediction.
You did mention, seed, which I haven't seen available anywhere else (in my limited experience) and it can explain a reproducible inference result, but I feel like this matches the "letter of the law" meaning of deterministic rather than the "spirit of the law".
If you send the seed along, you can have a non-zero temperature: when it's time to select tokens, a PRNG will be seeded and used for the selection.
Using a PRNG and a seed, you can have paeudo-randomness and determinism. The pseudo-randomness is the key piece, it’s random to the extent that it cannot be distinguished from true randomness in polynomial time classically.
Perhaps they just used the exact same random seed everywhere?