Comment by 8note
12 hours ago
if it puts a high confidence value on a wrong answer, thats still hallucinating, no?
llm hallucinations are high probability tokens that are incorrect vs the real world
12 hours ago
if it puts a high confidence value on a wrong answer, thats still hallucinating, no?
llm hallucinations are high probability tokens that are incorrect vs the real world
No, I don't believe so. Hallucinations are not "high probability" in a real sense. They are an artifact of the random walk the inference algorithm takes, which causes it to latch on to and chase attractors in the noise. This random walk behavior is necessary for chat interfaces to be useful, but are less critical to typed output predictors. I'm guessing they found some optimization that is possible if you give up caring about chat.
Correct, they have not made a universal all-knowing omniscient oracle, which is what would be required for "can't hallucinate".
That seems like a weird standard.
I would be happy enough with: only produces what it can verify with sources.
If you eg try to remember a court case (ie produce the reference via LLM token generation only), it's easy enough to check with your data whether it really exists. Similar for following links and other references.
If your data or sources are wrong, obviously your report about them will be wrong. But I wouldn't call that a hallucination.
There isn’t a single human in this world and hasn’t ever been that meets your happy-enough standard. Make of it what you will.
3 replies →
Not to be tooo pedantic, but a bot that assigned 0 confidence to everything wouldn’t hallucinate.
A calculator either gets the right answer or doesn’t answer.
It wouldn’t have to be all knowing as long as it knew perfectly what it doesn’t know
A quantum calculator answers in distributions.
1 reply →
What we would want to see if a confidence value that is in line with the actual correctness. If the value is 0.9 for 1000 different answers, then approximately 900 of those answers should be correct.
Yes, there is no magic sauce here that makes stochastic output binary if that’s what people are looking for.
Right and so maybe we should stop saying "can't hallucinate" when it can by definition.
It’s not what people are looking for, but what they wrongly claim.