Comment by solenoid0937
3 hours ago
Anyone that thinks that the hallucination rate is 59% has not actually used these models on a real project.
3 hours ago
Anyone that thinks that the hallucination rate is 59% has not actually used these models on a real project.
Hallucinations come up when asking knowledge bases questions, such as "In React’s Canary Fragment refs API, which FragmentInstance method returns a flat array of DOMRect objects for all children?" Or "Over what years did Roubini and Sachs examine 15 OECD countries when assessing trends in tax-to-GDP ratios?" While to me those may seem hyper-specific and unlikely to come up in a real context, students will absolutely ask questions similar to these.
I just asked the first question to gpt 6 sol medium and it replied:
> getClientRects() returns the flat array of DOMRect objects for the Fragment’s first-level DOM children. source: https://react.dev/reference/react/Fragment
2nd question it replied:
> They examined 1960–1986 for the 15 OECD countries. Source: https://www.earth.columbia.edu/sitefiles/file/about/director...
Are these hallucinations?
Are you suggesting that it's higher or lower?
IME on real projects you do need to be very careful with prompts about topics that are less likely be common in the training set.