Comment by jdiff
5 days ago
It's reasonable for tools to produce reasonable, predictable output to enable them to be used well. A tool can have awful, dangerous failure modes as long as they're able to be anticipated and worked around. This is the critical issue with AI, it's not deterministic.
And because it always comes up, no, not even if temperature is set to 0. It still hinges on insignificant phrasing quirks, and the tiniest change can produce drastically different output. Temperature 0 gives you reproducibility but not the necessary predictability for a good tool.
I don't think the "non-deterministic" accusation is a good one. Same as "hallucination", it's a bit of misdirection.
These LLMs are buggy. They have bugs. They don't do what they promise. They do it sometimes, other times they give garbled output.
This is buggy software. And after years and billions of dollars, the bug persists.
Yes we've all heard the AIs are not deterministic trope ad nauseam , but that's unrelated to my point.
MCMC is also not deterministic, and yet people learn how to use it well. Being non-deterministic is kind of the whole point of anything based on statistics. It's deterministic conditioned on the seed.
MCMC is reliably predictable. I believe I made it clear in my last comment that that was the goal, not actual run-to-run determinism which is achievable.