← Back to context

Comment by gpugreg

1 day ago

    > those things are not deterministic

Determinism was an explicit goal of DeepSeek-V4. From their paper: https://arxiv.org/html/2606.19348v1#S3.SS3

    > we implement end-to-end, bitwise batch-invariant, and deterministic kernels with minimal performance overhead

Of course, providers may not implement deterministic inference for various reasons, but it is possible.

Most people are not even referring to CUDA batching nuances.

They think that sampling is an inherent part of Transformers.

Even on this site, it is regurgitated with confidence.