Comment by Terr_
18 hours ago
I know I'm a bit of a broken record here, but "sometimes it hallucinates instead of being factual" is a bit like "sometimes the Ouija board fails to reach the afterlife spirits, instead of channeling them."
Both falsely imply that there's a solvable mechanical difference going on between results people like versus results people dislike.
I hate neologisms like hallucinate. Terms of art are fine for cognoscenti, but are misunderstood by naive readers as implying aspects of intelligence. It's just statistics ma'am.
I find it's a perfectly fine word to describe the result. Humans do the same as our visual system samples a low amount of data points to construct the view we see. In effect we're always hallucinating with the difference being that we maintain high context to filter it to the correct hallucinations. This shows up in dreams where we don't maintain such context, or when context is largely changed. When I got back from a vacation where there were many small and larger lizards, I swear I saw one upon returning, but it turned out to be a similarly colored leaf moving in a scurrying motion.
Edit: "mis-remembering" is another good term as it's using its vast training to output tokens and sometimes it maps them wrong.
My problem isn't the word per se, but the way it gets used to suggest it is an exceptional case rather than standard.
Perhaps true. But if you are warning against the dangers of so-called "AI" - "hallucinate" is also suggestive of "dangerous", "insane", and some other strong negatives.
That's reaching for a positive in the negative to me, but undeniable. So .. I dunno you win!
I continue to respect Hinton, he uses and chooses his words carefully I think.
1 reply →