Comment by lebovic
6 days ago
I built one of the connected tools included in this launch (the Biomni HPC [1]), and I have spent an inordinate amount of my life working on this problem. (I also worked at Anthropic, but not on this product.)
As other comments have pointed out, this is for data science – but it's capable of more than making plots and writing papers [2]. It has integrations with many databases and computational tools, including a researcher's institutional cluster.
That alone is valuable. I founded a startup after struggling with this problem at a bio startup; integrating these tools and databases is hard and time consuming. If the only outcome of this product is that great APIs are built for LLMs, it will be a massive positive impact. Many databases used in computational genomics are still only accessible through FTP!
LLMs are particularly good at navigating these tools and databases. It's often very specialized, but straightforward, work that benefits from in-context skills. Seeing an early glimpse of my former customers – bioinformaticians – using LLMs to solve this problem is what led me to join Anthropic in 2024.
Also, this pattern isn't fundamentally constrained to data science: you can also integrate with a wet lab or a CRO for some kinds of science. This is what I'm spending my time on now.
This type of science doesn't solve everything, but it's useful in some niches. For example, progress on many rare diseases is bottlenecked by researcher attention rather than a fundamental breakthrough.
[1] https://x.com/phylo_bio/article/2029233694775624096
[2] In comparison, OpenAI's science product – Prism – was effectively a LaTeX editor they acquired with Crixet.
Connecting AI directly to the data sources (instead of just asking it to provide code that I run locally for myself) can get quite complicated in terms of meeting institutional policy, applicable law, data access-storage requirements (e.g. NIH data repositories), and can require legal agreements between institutions and the AI provider.
I cannot touch. At least not yet.
If you put your data in Snowflake then there is a built in AI (ok it’s Claude) that can access the databases. This sidesteps a lot of the issues in that the data is clearly already with Snowflake.
Can you speak to what makes this different from simply including or configuring various agent skills? Or is it simply the combination of lots of helpful defaults that makes this product useful?
I can't speak for Claude Science, but I prefer using Biomni as an agent for bio over Claude Code with a custom setup because a) Biomni stays on the frontier for bio, b) it has a config that just works and skills I trust are correct, and c) it has better built-in abstractions for long-running sessions.
As a concrete example, computational biology jobs sometimes run for hours on the Biomni HPC. When they're done, the session needs to reawaken, process the results, iterate, etc. You can implement something like this with agent callbacks, but it's not as straightforward.
This repeats many times for many integrations, so it's just simpler for me to use an agent that's built for exploratory bio and already has all of this. Claude Science has some of these features, so I imagine they're aiming for something similar.
The FAQs at the bottom of the page answer your question.
The FAQ was exactly why I asked the question, since it made it seem like the answer is no.
How do you validate this kind of work to weed out any confabulating by the LLMs?
When you set up your Claude Science instance you can see that they're connecting to crossref, semantic scholar, pubmed, ArXiv, FDA. They instruct the LLM to validate citations.
My testing with this technique indicates that method they seem to be using (rag with an instruction to check sources) will reduce the confabulation rate for citations from the base rate 50-60% for regular models (e.g. regular Claude) to 5-15% (depending on how they implemented it). On the one hand this is way better. On the other hand it's just good enough that your spot check will look good and your work will still contain hallucinations (which is probably worse than obviously bad).
Getting to zero confabulation would require a different process. (stand-alone validation engine running in parallel in real-time which is hard but not impossible.)
I assume they do hallucinate, just like with coding or finding vulnerabilities.
You can try to minimize it (e.g. with a reviewer agent, which Claude Science and Biomni have), but nothing is perfect, so I limit autonomous work to verifiable problems and review it.
Honestly, this is how all AI should be used, in most non-trivial scenearios
i love how gp posted a glowing review and then dipped out.
After spending years on a problem, it's exciting to see it start to get more attention and move towards being meaningfully solved.
But I try to limit my time on HN, and I thought someone who works on Claude Science might respond to this thread later.
sorry, did they owe you and any other poster something?
4 replies →
Previously integrated Biomni into our intelligent workspace:
https://blog.codesolvent.com/2025/07/ai-assistant-with-biome...
Happy to chat if intrigued.
I'd really like to see much better visualization from Claude Science at some point. Educational-esque, with full threejs + shaders scenes over just these plots and protein/chemical structures. This for a lot of papers in the literature review would be awesome.
Sounds like the perfect use case for some kind of framework where you have a local LLM (that can run on lower spec hardware) collaborating with the main LLM to optimise latency and all the other niche and legacy use cases ?
Thank you for this summary. Especially interested about the wetlab & CRO tie-in. What is meant by a ‘researcher’s institutional cluster’?
> https://www.anthropic.com/_next/image?url=https%3A%2F%2Fwww-...
Did it produce this absolute clanger of a visualisation that still proudly sits on top an anthropic blog post?