Comment by nonethewiser
9 hours ago
This is an interesting way of thinking about it. I generally agree. I especially agree that anti-AI sentiment partially comes from miss-using it. However:
Determinism isn't a requirement for 100% correctness.
A Las Vegas algorithm is randomized, non-deterministic and guarantees 100% correctness [0].
The execution can be different every time but the result will always be correct. determinism does not lose accuracy. It does lose time predictability.
So if your problem with AI is accuracy, then in theory your problem is just premature stopping.
A Las Vegas algorithm requires that you have a deterministic test that can definitively determine the correctness of an intermediate result. So what you're saying is that what it takes to make LLMs give 100% correct results is having a human between the LLM and the user, who's capable of re-prompting on incorrect answers from the LLM. Well, if the human is there, why not just ask the human? What value is the random number generator adding?
Like the GP said, the point of determinism is that you can trust the correctness of the results, without doing any checking. Solved problems stay solved.
> So what you're saying is that what it takes to make LLMs give 100% correct results is having a human between the LLM and the user, who's capable of re-prompting on incorrect answers from the LLM.
No, that's not what I'm saying. Im saying determinism isn't required for correctness. And Im not saying that models are already perfect other than prematurely stopping. What I am saying is that non-determinism doesn't mean they cant be 100% correct.
Besides, humans baby-sitting LLMs is not even implied by the misinterpretation of what I'm saying. What's implied by that is humans needing to give the LLM explicit success criteria from the outset. Which is totally reasonable.
> Like the GP said, the point of determinism is that you can trust the correctness of the results
Well, no. Now you are waaaay overstating determinism. Deterministic results might be incorrect.
>What's implied by that is humans needing to give the LLM explicit success criteria from the outset.
But the LLM is non-deterministic. LLM(x + y) or even LLM(LLM(x) + y) is just as non-deterministic as LLM(x).
>Well, no. Now you are waaaay overstating determinism. Deterministic results might be incorrect.
Obviously? But that's still is the reason why deterministic computers are useful. If you could enter a proven-correct program into a computer, run it, and get back a correct result only some of the time, that'd be okay maybe for some things, but we couldn't have built everything we have on top of that. It's the same reason why deductive reasoning is useful. Once you've proven something true, you're done, it's proven. No further work is necessary.
The economics of it (token cost) means, however, that what will be chosen most often is the barely sustainable minimum level of quality, aka race to the bottom. AI is more cost-sensitive in that way than humans caring or not caring about making things robust and correct used to be.
For sure, that's why I say in theory. Also models aren't perfect. You can't plug in an explicit target and ensure it doesnt stop prematurely and expect 100 accuracy. But that's not because it's non-deterministic, since clearly non-deterministic systems can be 100% accurate.