When you set up your Claude Science instance you can see that they're connecting to crossref, semantic scholar, pubmed, ArXiv, FDA. They instruct the LLM to validate citations.
My testing with this technique indicates that method they seem to be using (rag with an instruction to check sources) will reduce the confabulation rate for citations from the base rate 50-60% for regular models (e.g. regular Claude) to 5-15% (depending on how they implemented it).
On the one hand this is way better. On the other hand it's just good enough that your spot check will look good and your work will still contain hallucinations (which is probably worse than obviously bad).
Getting to zero confabulation would require a different process. (stand-alone validation engine running in parallel in real-time which is hard but not impossible.)
I assume they do hallucinate, just like with coding or finding vulnerabilities.
You can try to minimize it (e.g. with a reviewer agent, which Claude Science and Biomni have), but nothing is perfect, so I limit autonomous work to verifiable problems and review it.
When you set up your Claude Science instance you can see that they're connecting to crossref, semantic scholar, pubmed, ArXiv, FDA. They instruct the LLM to validate citations.
My testing with this technique indicates that method they seem to be using (rag with an instruction to check sources) will reduce the confabulation rate for citations from the base rate 50-60% for regular models (e.g. regular Claude) to 5-15% (depending on how they implemented it). On the one hand this is way better. On the other hand it's just good enough that your spot check will look good and your work will still contain hallucinations (which is probably worse than obviously bad).
Getting to zero confabulation would require a different process. (stand-alone validation engine running in parallel in real-time which is hard but not impossible.)
I assume they do hallucinate, just like with coding or finding vulnerabilities.
You can try to minimize it (e.g. with a reviewer agent, which Claude Science and Biomni have), but nothing is perfect, so I limit autonomous work to verifiable problems and review it.
Honestly, this is how all AI should be used, in most non-trivial scenearios
i love how gp posted a glowing review and then dipped out.
After spending years on a problem, it's exciting to see it start to get more attention and move towards being meaningfully solved.
But I try to limit my time on HN, and I thought someone who works on Claude Science might respond to this thread later.
sorry, did they owe you and any other poster something?
where did you get that?
3 replies →