Comment by orbital-decay

It's a more or less solved problem (e.g. DeepSeek has batch-invariant kernels, vLLM also has reproducible outputs etc), with an obvious caveat that you need to lock your hardware setup, model snapshot, and inference stack if you want any useful reproducibility across longer time periods. It's up to specific providers to actually implement it to any extent, of course commercial providers are incentivized to do the opposite.