Comment by robrenaud
2 hours ago
Hallucinate -> do an experiment -> see it fails, try again. Hallucinate -> do an experiment -> it works, model innovated.
Agentic models brush up against reality, this gives a way around the hallucination problem.
Here is a recent talk showing that hallucination and discovery are actually positively coupled. https://www.youtube.com/live/ZNlZsI9kBm4?si=nhn4ancXu7s6qtom...
Ground breaking thought (ML Researchers Will Hate Me For): (some) hallucinations are actually creativity.
Creativity unbound by reference is not even dada (which was reactionary) it is noise
Because - it's the part people are having hard time to accept - creativity was never the hard part. It's randomness. The hard part is navigating the narrow border between "not enough random, therefore boring" and "too much random, therefore unhinged/insane".
That border is determined not as much by ground truth, as by people's sentiment. As such, many famous artists and creatives never succeed navigating - their contemporaries would call them crazy, and then decades or centuries later, the border shifts, and suddenly they're remembered as greatest creators of their era.
1 reply →
In practice, hallucinations have only mitigated within verifiable domains. The vast, vast majority of human activity is unverifiable, or at least significantly less verifiable than mathematics.