Comment by SuperV1234
4 hours ago
What is the actual problem with the lack of determinism?
Why is auto-completion a problem if it's done with AI?
4 hours ago
What is the actual problem with the lack of determinism?
Why is auto-completion a problem if it's done with AI?
it's in the definition of the word. you can not determine what the LLM will do.
anything done with AI is a problem because it is essentially unpredictable. auto-complete is on the fence because you presumably are still able to pay attention that it completes what you want but it depends on how diligent you are when working and how much i trust your diligence.
You are still dodging the question -- what is the problem with not being able to determine in advance what the LLM will do?
Even then, you can clearly see that the LLM will try its best to follow the instructions. The result might not be 100% predictable, but it is somewhat predictable depending on the task.
After the LLM does what it has been asked, you can review it, iterate on it, test it, and so on. And if you're trying to make anything worthwhile, you will do so.
Lack of determinism is not a practical concern.
anything unpredictable is inherently untrustworthy and requires extra effort to review.
Lack of determinism is not a practical concern.
it is to me. it's a knockout criteria. it is the only reason that keeps me from using LLMs for coding. nothing else is as serious an issue to me as this.
here is why: i tell the LLM to build something with requirements A B C D and E. it builds, i review and i find A B and D are good, C and E are broken. i tell it to fix them, it does, so C and E are fixed, but now A is broken. i tell it to fix that, and i have to keep iterating until i find a combination where everything works. in every iteration any part can randomly break, so for every iteration i get changes all over the place. they never are confined to the issue i pointed out. i have to review the whole thing every time. that's what i mean by lack of determinism, and that is a serious practical concern because instead of getting done in two or three iterations it requires dozens of them. see my related replies elsewhere. i just don't want to work that way.
2 replies →