Comment by adastra22
13 hours ago
As someone working in science, this belief confuses me. How (by what means) do you think Fable 5.1 will be able to make further progress in scientific domains? The problem with science is that there is no agentic harness. The agent can't test things. At best it can hallucinate something and ask if that hallucination "makes sense", but this doesn't work in science.
Sounds like a very narrow view on what constitutes science. There are many fields of science where there is existing data against which new ideas can be tested without additional 'real-world' measurements. Newton's theory of gravitation relied entirely on pre-existing astronomical data for which there was no existing unifying theory. He made progress by putting forward a theory which explained that data. Now you can argue that it's not really science unless you include the original data collection and subsequent real-world measurement validation steps. But I'd be comfortable saying that Newton was indeed a scientists and did make progress in science despite only doing what some might say is the 'middle' part of the process. There are plenty of modern analogs where work like this sits out there waiting to be done using existing data.
I suggest you to give a look to the MCP protocol for hardware that is being proposed by Anthropic. The hardware will be the next harness of LLMs, they will be able to operate machines to reinforce their theories.
I still think that a major problem is that biological processes are not “fast” as coding, but they are verifiable. If during post processing we are able to give enough harness to test and verify this kind of environment (maybe via simulation and real data) we will for sure achieve incredible performance also in this domain.
The field I'm in requires millions of dollars of very sensitive (fragile) capital equipment, and latencies measured in weeks and months. Agents tend to move fast and break things, which matters less when you are writing code under version control.
Have you worked with agents on tasks with high capital and long latencies?
Having worked with Fable 5, the feeling I get is that it's fairly capable of accounting for these tradeoffs and will depend fast more time on planning and testing.
At the end of the day though, with horizons like that the best use of an AI is to get it to help you with those things, not so much delegate fully.
1 reply →
> I suggest you to give a look to the MCP protocol for hardware that is being proposed by Anthropic. The hardware will be the next harness of LLMs, they will be able to operate machines to reinforce their theories.
Yeah, that's called an API. Again.
The actual hard problem that this hand waves is making (and funding the making of) hardware to reliably do the things you need it to do.
Great news, then! TFA: "Last week, we previewed the Model Hardware Standard, which allows Claude to directly and safely operate laboratory equipment."
The bottleneck in science isn't ideas or human work speed. The bottleneck is resources and time to get experimental results.
LLMs, even in control of lab equipment, address neither of those.
Thank you. This is what I'm driving at, that most of the AI and software devs here seem to be missing. Intelligence is not, and never was the bottleneck for most science/hard tech. Full AGI gets, at best, a small productivity improvement, which over long periods of time does have compounding effects. But this isn't a singularity hard-takeoff inflection point.
How much lab equipment is automatable though? There's definitely some in biology, but if you're doing fundamental research it's 99% stuff you are building yourself with your own hands. Robotics is a long way from being able to do any of that.
When I was doing research (physical electronics, lasers, fiberoptics and sensors stuff), lot of time was spent just writing all sorts of DAQ and processing code. So all this LLM stuff would have been really useful. There's a lot of data collection, data processing in the lab that require all sorts of ad hoc scripts and stuff. That was many years ago, but I would be surprised if the current crop of researchers are not using these things. And if they are not, then they are just not serious.
>but if you're doing fundamental research it's 99% stuff you are building yourself with your own hands.
You can do LLM->3D Printed models now. The drone can fly in and pick them up and bring them to the location you want. They can assemble structures. All automated, all LLM driven.
Things are changing. What was true, no longer is.
1 reply →
Outside of biology, very very little.
that might indeed be a problem for all the pulp-producing labrats of STEM in southern europe and the third world.
However I think this area has so much decoupled from industry and solid research institutions that they might not notice at all (beyond their use of AI-generated slop to augment the slop they already produce)...
When I looked at “Claude Science” which is a beta, separate desktop app, I came away with the impression that it was mostly for biology and a bit of chemistry - presumably there’s some value it can get from consulting obscure literature and uniting disparate threads of already-known stuff, but since I don’t work in either field I can’t speak much more to it.
> As someone working in science, this belief confuses me. How (by what means) do you think Fable 5.1 will be able to make further progress in scientific domains?
The same way it did in the previous versions: brute force.
I don't believe that LLMs have any particular intelligence we don't, but there's an endless list of problems we either don't have bodies to throw at, or the bodies we can throw at it, don't have such a huge large context to crunch problems.
What LLMs will always intrinsically fail at is showing us genuine new intuitions. The technology is about predicting the next plausible token/sentence.
They will not revolutionize human knowledge, but they can definitely widen it a lot.
> They will not revolutionize human knowledge, but they can definitely widen it a lot.
I am generally quite enthusiastic about all this, but my biggest fear is that we will not recognize the extreme need for more scientists at a time when there is so much more science to be done. The rate of scientific understanding must keep pace with the amount of science being output, both for verification and further discovery. It's a pipelining issue, and I predict a stall in the bits that require the (currently rare) people who know what they're doing.
We are not limited by intelligence, or bodies.