Comment by r3trohack3r
5 days ago
Most (all?) AI models I work with are literally deterministic. If you give it the same exact input, you get the same exact output every single time.
What most people call “non-deterministic” in AI is that one of those inputs is a _seed_ that is sourced from a PRNG because getting a different answer every time is considered a feature for most use cases.
Edit: I’m trying to imagine how you could get a non-deterministic AI and I’m struggling because the entire thing is built on a series of deterministic steps. The only way you can make it look non-deterministic is to hide part of the input from the user.
This is an incredibly pedantic argument. The common interfaces for LLMs set their temperature value to non-zero, so they are effectively non-deterministic.
From the good old days: https://152334h.github.io/blog/non-determinism-in-gpt-4/ (that's been a short two years).
Unless something has fundamentally changed since then (which I've not heard about) all sparse models are only deterministic at the batch level, rather than the sample level.
Even after temperature=0 I believe there is some non-determinism at the chip level, similar to https://stackoverflow.com/questions/50744565/how-to-handle-n...
> I’m trying to imagine how you could get a non-deterministic AI
Depends on the machine that implements the algorithm. For example, it’s possible to make ALUs such that 1+1=2 most of the time, but not all the time.
…
Just ask Intel. (Sorry, I couldn’t resist)
So by default. Its non-deterministic for all non power users.