Comment by Animats

> Did we get a grip on this problem?

That's the right question to ask. For a while, it seemed that hallucinations went down as models got bigger. That may only have been because, with a big enough model, the desired data might be in the model, somewhere, which would keep the model from making up something. That's the brute-force approach to the problem.

This new article indicates that trimming down the model by pulling out seldom used info makes the problem worse again.

If LLMs had reliable "I don't know", and access to search engines, much smaller models might work.