Comment by ivan_gammel
6 hours ago
You are right with the analysis, but wrong with the conclusions. Yes, LLM „thinking process“ is kinda non-deterministic in a sense that it does not follow logical reasoning and will not produce logically correct results in 100% cases. It has an error margin.
However, error margins are in the center of any engineering discipline. We cannot produce things measured with 100% accuracy. This is accepted fact. The focus is always not on eliminating errors, but on reducing them to acceptable minimum. With LLMs we should not expect an ideal logical thinker, but a process that may error sometimes, and we must design quality controls instead that push LLM outputs within acceptable margins. And it can work.
Yes but the key here is doing proper risk assessment. "What is the consequence if the LLM gets this wrong?" "How do we verify the output?" "What are the legal ramifications for using the LLM in this way?" "Who is responsible when the LLM fails?" "Whats the expected accuracy here?" etc.
In the current AI mania, there's a lot of due diligence simply being ignored. Plenty of "Well humans make mistakes too!" going on here on HN too.
The due diligence not being done is people putting cats in microwaves. It‘s not the dangerous part. The real danger is risk assessments coming to wrong conclusions, because it is still terra incognita. Talented engineers were in this situation before, doing mistakes with cars, airplanes, buildings etc.
No, I'm sorry but I think thats a cop out. The fact that LLM are stochastic and can give incorrect answers is not particularly difficult to comprehend, and the risks that fall out of that are reasonably understandable. The issue is entirely down to bad choices by the people driving LLMs, because they are engaging with what they wish LLMs do instead of what they actually do.
1 reply →