Comment by gpugreg
19 hours ago
> those things are not deterministic
Determinism was an explicit goal of DeepSeek-V4. From their paper: https://arxiv.org/html/2606.19348v1#S3.SS3
> we implement end-to-end, bitwise batch-invariant, and deterministic kernels with minimal performance overhead
Of course, providers may not implement deterministic inference for various reasons, but it is possible.
Most people are not even referring to CUDA batching nuances.
They think that sampling is an inherent part of Transformers.
Even on this site, it is regurgitated with confidence.