Comment by paxys
19 hours ago
Remember that LLMs are still computer programs, and so are inherently deterministic. A model given the same input multiple times will always produce the same output. The randomness is added on top. This is why LLM-produced text, websites, images all seem so generic.
It's likely that multiple agents doing a certain task all independently thought "let me try writing on this website".
> This is why LLM-produced text, websites, images all seem so generic.
No, it is not why. That's not inherent to the LLM architecture at all but appears after RL training. Base models don't have any problems with genericness.
What? No. Have you ever worked with programs that do floating-point math on a GPU? It's not deterministic, definitely across platforms, and often within a single platform, too.
> Have you ever worked with programs that do floating-point math on a GPU? It's not deterministic, definitely across platforms, and often within a single platform, too.
Floating point math is 100% deterministic, but different hardware/OS have different but deterministic behavior in some corners. The same code run on the same hardware with the same inputs (including access to timers, peripherals, etc.) will behave the same way, unless you're talking about cosmic rays flipping bits or something.
It's certainly possible to write LLM inference on the GPU in a deterministic way, but it's somewhat nontrivial and trades off against performance, so by default most LLM inference engines aren't deterministic even at zero temperature. The classic post about that is https://thinkingmachines.ai/blog/defeating-nondeterminism-in...