Comment by meindnoch
6 days ago
>I've been pondering on something related: can an LLM be a chat?
A chat?
>I am not talking about an "AI agent", but really a model that we call in a reproducible manner.
An LLM is just as deterministic as any other computer program. For identical inputs (which includes the PRNG seed) it produces identical outputs.
>compressed version of the git kernel
The git kernel, got it.
>But would this count as "compression" at all?
Yes. The decompressor is several tens of gigabytes though.
>An LLM is just as deterministic as any other computer program. For identical inputs (which includes the PRNG seed) it produces identical outputs.
This is not really true in practice because of multi-threading and out-of-order execution. Mathematically equivalent orderings of operations are not equivalent when dealing with floating point values, so most practical LLM implementations end up being non-deterministic.
"An LLM is just as deterministic as any other computer program" is not refuted by pointing out hardware limitations that would affect any other computer program implemented at similar scale (weather forecasts, or even just computing the average of a large stream of sensor readings).
I'm not really trying to 'refute' the original statement. Certainly, an LLM is just doing some calculations that can be done deterministically in principle. However, I think it's worth pointing out that there are practical barriers to doing those particular calculations both deterministically and efficiently. People who worry about LLM output not being reproducible aren't necessarily misunderstanding what an LLM is doing; they are responding to a real feature of most practical LLM implementations.
1 reply →