Comment by WithinReason
19 hours ago
Apparently it's news to some people. It also helps combat the "LLMs ArE nOt DeTeRmInIsTiC" complaints.
19 hours ago
Apparently it's news to some people. It also helps combat the "LLMs ArE nOt DeTeRmInIsTiC" complaints.
Doesn't help much
> It also helps combat the "LLMs ArE nOt DeTeRmInIsTiC" complaints.
Reality says otherwise, especially from those who run LLMs in production at a large scale. Hacks such as baking in the weights with tiny experiments doesn't disprove anything.
The same LLM boosters also continue to repeat one of the biggest misconceptions: "sEt TeMperaTurE tO zERo tO aChiEvE fUlL dEterMinIsm."
The accurate statement is that LLMs are not inherently non-deterministic. From a mathematical perspective, they’re obviously deterministic. But choices in the way we deploy them introduce non-determinism.