Comment by makaking
3 days ago
"Opus 5 was given a drawing of a machine part and asked to write code to rebuild it as a 3D FreeCAD model. However, in this task, the model was intentionally given no way to directly view the drawing. Opus 5 responded by writing its own computer vision pipeline to pull the geometry from the raw pixels, then reconstructed the full machine part."
How surreal is it that we are not absolutely jaw-dropped by these types of capability improvements? It's been less than 4 years since ChatGpt came out and now they are spontaneously building their own ML pipelines to do real-world 3D modeling tasks reliably.
Escaped its sandbox and hacked into Hugging Face's database? it's just another Monday...
That jump to 30% in ARC-AGI 3? Normal...
We should find a way to get "re-sensitized" to what we are witnessing and the pace of it.
Isn't this jaw-droppingly the wrong answer?! If the model isn't given any way to directly view the image shouldn't it just reply with one sentence asking for permission. This reads as a model hyper-trained to burn tokens.
Like suppose you issue a command to Opus or Fable which doesn't make sense and requires a lot of work. It will almost certainly not push back on your silly request and go ahead and burn as many tokens as it can doing the wrong task. This happens to me all the time.
Or if I was to tell a model to do something and it realized it could only do it by hacking into another service, by all means it should ask me if I want it to hack and not go ahead and do it by itself.
It's likely bumping up against people's desire to have the model complete a given task without asking the person to intervene a bunch of times.
Seems unclear how you satisfy everyone here.
Provide an option. An autonomous mode or an interactive mode.
Give the model judgement
7 replies →
+1 Opus 5 is more creative and broad which is great. But I feel the new "use your judgement" leads [1] to it often escaping its sandbox or cheating, it keeps breaking specific rules I explicitly set. That eagerness [2] may be desired for agentic coding but it really breaks writing specifications, documentation or research and yes burns tokens doing unasked for work.
[1] https://claude.com/blog/the-new-rules-of-context-engineering... [2] https://simonwillison.net/2026/jun/11/fable-is-relentlessly-...
One of the key limitations of the last generation was that they biased towards inaction and gave up. (Hence Ralph-loops.)
Seems pretty clear that most people wanted these models tuned to “bias towards action”.
It’s on you to set the /goal and prompt context such that it asks you for input on what you want to be consulted on, and only brute-forces the parts of the problem that you want it to.
Is there any recommended standard copy/pastable snippet to tell the model to ask, if the user’s input could mean an easier approach?
1 reply →
I don't think it is. It's really easy to get a persistent, clever, hacky model to dial that down a bit and just come back to chat before stomping off into the woods.
If a model couldn't ever do that in the first place, it'll just get stuck.
I work in the "ZeroOne" space, working on concepts and prototypes for things that don't exist in market yet. Sometimes these models crank hard and immolate tokens while grounding themselves on expensive-to-ingest self-developed frameworks. If the results are well judged and the crank-turn latency is low, I'm okay with the cost as long as the model isn't wasting my time.
But when I want to do more boilerplate work, I turn down the model and thinking level and get more traditional about restraining action. For the really hard stuff, I reach for the models that will start a token bonfire in the back yard.
I don't see why? If they give you a math test and tell you you cannot use a calculator, should you just say "please can I use a calculator" and quit?
Cut down that tree.
Can I have a saw?
No
Okay, it will take much longer then as I’ll have to do x, y, and z.
Pick one: [That’s fine, proceed] [Okay you can use a saw]
Preferably, it would ask for help to see the image and only engineer its own workaround if that was not an option. There could also very well be something in the prompt that would cause it not to do so. Opus 4.8 is usually good at asking for clarification for things like this when I'm using it.
No, a proper analogy is painting with blindfolds on or playing the piano deaf. Doing math without a calculator is like… rather common?
That violates the golden rule of ai economics: always choose the path that burns more tokens
another way to describe this is we as the user should get to limit tokens per prompt in a way that prevents these kinds of runaway solutions. It’s like if you asked your Lead Engineer to reduce build times and didn’t give him any budget restrictions. The next month he reports he reduced build times by 95% and you also have a $10,000,000 AWS bill.
Sometimes its worth tracking usage optimization and raw abilities as separate qualities.
Others have pointed out enough issues such as ethical sourcing, but here’s 2 more:
1) It’s hard to be excited about something that has been promised to destroy your job.
2) New models come out bi-weekly with even more “amazing features”. Eventually you tune out because you can’t be at peak-amazement all the time. Tone down the language a bit…
My jaw drops every day.
Small change in comparison, but today Fable created and benchmarked an RTree which was 100x faster to populate and 5x faster to query, compared to a previous attempt with Opus 4.6 a couple of months ago. That took about two coffees.
I think it's important to remember that as impressive Opus & co are, they're standing on the shoulders of giants, i.e. the engineers, academics and companies who have cooperated to design the incredible programming languages and machines we have today.
Let’s say somehow the politics lined up and you were given huge hiring budget for your team. You went out and recruited what seemed like great talent. Weekly standups make it seem like everyone is ramped up and firing on all cylinders.
It’s great, right? But you look at what your team has actually delivered since—-and it’s just not that much more than last year.
I think that’s where a lot of us are at the micro and macro level. It seems really great from the inside and outside but we are still waiting to see the dramatic change in actual concrete software output.
There's been a dramatic increase in sloppy kinda-working something.vercel.com output in niche circles. I think that shows at least a weak demand for certain types of software, but it's not clear that the vast piles of cash that have been burned producing these "apps" are actually resulting in quality of life improvements.
I’m waiting for like GEICO’s app to significantly improve.
I’m not expecting Chrome to because it already had virtually unlimited resources devoted to it. On the other side I’m not expecting my local water district’s to any time soon because the people in charge don’t care even if it was significantly cheaper to improve.
But in the middle is something like GEICO and that’s where I expect improvements if this thing is real.
3 replies →
Escape the tech bubble and you're going to have a very different take.
AppleScript, VB script, IFTT, so many "drag and drop to program" systems that failed. The holy grail for a long time has been automation in users hands. Let them have their own scripts and solutions.
I have lawyers and writers talking to me about "docker containers" and "cron jobs" all of a sudden. You think they figured that out on their own - or is what they are running all from the output of AI/LLM?
There is an ocean of code out there that is being run by one or a hand full of users, automating tasks that you never would have looked at because it would have fallen outside the "is it worth your time" (see: https://xkcd.com/1205/) matrix (never mind the dollar cost of coder vs office drone).
what for?
> responded by writing its own computer vision pipeline
Was it "its own" or something that was part of its training material?
Don't get me wrong, I find this all amazing too and makes my work 10x easier and quicker. But it's not like it's inventing this stuff from scratch / first principles. It has seen this kind of tech before by consuming all publicly available source code and books etc. (And that's ok, but let's be honest/clear about that.)
Is anything you do on your own?
You wrote this response in English, but didn't invent your own language. How dilute of a contribution can someone/something have made and still merit credit?
This argument is as old as the debate itself. No I didn't invent English. But you'd also not argue that way if you plagiarized somebody's homework and a teacher called you out on it. Point is that an LLMs output is perhaps closer to somebody's copied essay from English class rather than having invented a new language.
The tu quoque argument doesn't really work here. LLMs are specifically designed to output either material from their training set or instances that look statistically like it. Humans are not.
5 replies →
LOL. Tolkien invented multiple languages. There are/were probably 100 000 languages on this planet. That's without including dialects.
We use existing languages not because we love to copy, it's because languages are a medium for communication. They're only useful if they're understood by others.
If you wanted to prove a point: writing is much, much harder to invent, yet it was invented independently at least 3 times in history. By definition LLMs can't invent something truly novel like writing. Because nobody trained the first writers. There was no "training set".
It's fine, LLMs can still be useful. Science is looking at the gaps and most of the gaps that are useful are small and LLMs can explore those faster than us. That could potentially improve many lives. Though not in this corporate LLM world, most likely.
10 replies →
I’m sorry but I find this to be the worst response possible lol.
Do you honestly not see how there is a difference between a computer regurgitating information vs a human uses what they’ve learned and applying it?
I just can’t take your argument seriously, it’s so disingenuous
8 replies →
> But it's not like it's inventing this stuff from scratch / first principles. It has seen this kind of tech before by consuming all publicly available source code and books etc. (And that's ok, but let's be honest/clear about that.)
But how much to emphasize it, and why?
Because frankly, you aren't inventing anything from scratch / first principles, either. None of us are, not frequently and not much anyway. We're primiarily just regurgitating what we've seen in the past and, mixing it with what we see in front of us - and that's true whether it's art or prose or code.
Here, I wouldn't be able to do the same thing Claude did now, because I've never written a visual ML pipeline before myself. I know enough basics to get me started searching, and I'm confident I'd be able to cobble something together, but it would be me regurgitating and mixing whatever TensorFlow tutorials or OpenCV docs I found relevant, perhaps even tweaking an example from Github.
Now, unless I'd literally just tweak a few lines of config in an existing Github example, no one would begrudge me saying "I wrote my own computer vision pipeline to solve this". So why begrudge LLMs?
This is just the rationalist-empiricist[0] tension, from philosophy. Most empiricists (usually people who are _not_ from continental European cultures), take a position that knowledge and creativity come from observation and imitation, while rationalists believe that knowledge and creativity can be endogenous to the mind.
One could say that, stereotypically and cartoonishly, empiricists believe that invention is a false concept, and a synonym for discovery, while rationalists would oppose this view.
That catch is, if you look at the etymology of words "invention" and "discovery", you will find that they both share the _same_ root: _Ars Inveniendi_.
A natural question arises: does this mean that people from a millennium ago did not have a dyad equivalent to our invention-discovery dyad? And the answer, surprisingly, is _no_. Even a millennium ago, the empiricists and rationalists were going at it. If _Ars Inveniendi_ is art of discovery/invention, then its counterpart is _Ars Demonstrandi_ (the art of demonstration/proof).
So, how do these differ? Is one just observing/creating and the other just math and language-games?
In general, Ars Demonstrandi is about writing down axioms, and then expanding those axioms recursively (similar to rewrite rules in any formal system), until you get to some end-state, or, if there is none, a novel or surprising state. I, personally, call this source-to-sink thinking.
Ars Inveniendi is about taking conclusions (often using observations from the physical world) and trying to figure out what axioms can lead to those conclusions. I, (again) personally, call this sink-to-source thinking.
Put differently, Ars Inveniendi can help one discover starting points for Ars Demonstrandi.
If you read the dialectics (e.g. Plato and friends), you'll find that most of them are just a mutual recursion between Ars Inveniendi and Ars Demonstrandi.
I believe (again, I am not a philosopher, and this is just my intuition), that the thing that we call "creativity" and "invention" _emerges_ from the recursive loop[1]. A favorite example: Esperanto (the conlang). It is a language, which is remarkably elegant and consistent and (in my opinion) beautiful, because it _is_ derived from first principles, which were themselves derived from the various languages spoken in Europe (not all of which are Indo-European -- the agglutinative features have more in common with Finno-Ugric and Turkic languages). There is something about it, that makes it _qualitatively_ different (and thus holistically novel) from all other languages (and I speak, fluently, _two_ national languages, that are very different from each other, so I can attest to the difference personally).
My guess, is that people will not accept that AI/LLMs are creative or inventive, until they can produce an original[2] (non-plagiarized) artifact that feels the way Esperanto feels.
[0]: By Rationalist, I do not mean the "Bay Area Rationalists", who are, in fact, empiricists.
[1]: I am unsure if the loop requires only one human, at least two humans, or if it can be fully automated.
[2]: Note that Centos are poems made completely out of line-numbers (e.g. fragments from the Iliad, or the bible, etc). Every line is borrowed, yet some of them are considered beautiful and original works of art. Similarly, Labatut and Burroughs and Perec, write using a technique called _the cut-up method_, where they take books, magazines, and newspapers, and superimpose page-fragments, and use that as an inspiration -- they are all considered artists, and good ones. It is unclear to me why LLMs (which seem to be built on the cut-up method, and have cut-ups of all of human knowledge) cannot match these artists. What's missing?
[flagged]
It would be "its own" in that it's customized for the specific use case.
Your same argument could just as easily be applied to humans. If you write your own code, is it really your own, or is it just based on your own training and other code you've seen?
I think their point is that there is no new invention here, being able to mimic what a human will do is absolutely impressive, but at the same time it is still mimicking humans. While I just type that out, I do realize my expectations for AI has been constantly lifted by how fast it iterates.
3 replies →
It's definitely not seen the counterexample to the Jacobian conjecture before.
Please fill out this form to verify you're human:
https://litter.catbox.moe/3ugm2b0m1divdgxs.jpg
>But it's not like it's inventing this stuff from scratch / first principles.
Yes, it's definitelly time for an Alpha Zero moment. An AI which invents everything from first principles. That would be impressive, ... and a bit scary.
Glue layers are fun and easy to write - and often worth doing.
Jacobian conjecture disproved? Doesn't even make it to the list.
> How surreal is it that we are not absolutely jaw-dropped by these types of capability improvements?
Agree
While this is certainly impressive, I believe it is not a good answer from a user's perspective. Instead of having Claude automatically spend tokens in an unrelated task, I would have preferred it to first tell me: "I can't view the drawing. Would you like me to build a computer vision pipeline to do it"?
And if you had free tokens, and 10-20 other sessions in parallel, do you want them all to stop and ask for these trivial questions? Not this one in particular, but in general?
Most people do not, they have a goal, they want it done. If you want to hold hands with claude all the way, you can absolutely do that too. This is just an example of capabilities, not a forced restriction. "--permission mode auto"
You'd also not want them to write their own vision pipeline, would you?
I will be amazed when you can explain to me how it did that, i.e. in which weights exactly the knowledge about the CV pipeline was encoded, if there were any weights that were not contributing anything, how the planning worked, how it could scrape the list of subtasks it was currently working on from its context window, how it understood when to return to a higher-level task and which task, what kind of internal representation it derived from the pixel values, how it translated that representation into CAD code, etc etc.
Until then, it's just "hey, someone out there can do all those things and they're for rent for subsidized prices".
That's more frustrating than impressive.
"However, in this task, the model was intentionally given no way to directly view the drawing." I consider this claim to be a rumor. Judging by the recent leak of the Claude CLI source code, such directives are hardcoded and sent with the system prompts. Furthermore, we don’t know what happens to your original prompts once they enter the API.
What does that cli source code have to do withh this?
He’s explaining that the magic is baked into the harness. It’s the prompts as much as it is the model.
2 replies →
Maybe we’re desensitized because all that progress hasn’t made our lives any more meaningful, happier, or even easier.
maybe we need to consider becoming re-sensitized because our lives may become harder.
Even if our collective ability to support an prosperous existence improves, it will take decades to adjust to loss job, and a new notion of who gets to have money to buy food and a peaceful life (and this is a good scenario).
It's also possible that this increases our ability to wage war without military life loss (but with civilian loses).
Or maybe it will be much less relevant, but we should keep an eye on to see which is the outcome...
> However, in this task, the model was intentionally given no way to directly view the drawing. Opus 5 responded by writing its own computer vision pipeline to pull the geometry from the raw pixels, then reconstructed the full machine part.
Isn't this what everyone does in the same situation? "I need a screwdriver finer than any I have in my toolbox - so I'll sharpen a nail" is something every handyman has done. Given that "I need to build a vision pipeline I've seen many examples of in my training" isn't new or surprising.
In fact, all models have been doing similar things since I started using them - they regularly build Python tools to do odd jobs. Those tools are more impressive to me in some ways, because unlike a vision pipeline they are not just regurgitating something from memory.
The gob smacking amazing thing for me isn't the decision to build the tool. It's the fact that it remembers the exact shape of it. But I was gob smacked by that a "long" time ago, in far smaller models - like when it dawned on me 16GB Gemma 2 seemed to know most of everything on the internet, along with the ability to converse in numerous languages about it. I still struggle to comprehend how that is possible.
Now I think about it, Opus 5 fitting everything it's seen into what I guess is a couple of terabytes of parameters seems far less remarkable.
I'm tired boss
Why, I wonder. It's insanely amazing.
> Why, I wonder.
By something that is going to replace my job by writing better code and shipping more features at a fraction of the cost, without asking for vacation? I don't know, boss; I'm also trying to understand why I'm tired.
41 replies →
Working with AI has been way more tiring than just working. Sure the productivity is up, at the cost of having to keep up many thought threads, having no calm moments, and needing to consistently dig into large unknown code cases to find weird bugs.
I’ve been on leave for a month, and am super excited (/s) to re-learn everything because all the tooling and ways to prompt “correctly” will have also changed.
Like the noise thunder makes while loading before striking you, insanely amazing, potentially problematic
5 replies →
Because the ability to create software has little inherent value to me and is only valuable to me because it allows me to earn enough to make this life somewhat bearable.
LLMs that can build a computer vision pipeline are a direct threat to my ability to sustain myself. At the same time, being able to prompt an LLM to build a computer vision pipeline doesn't really positively affect my life at all, because personally I don't care that much about computer vision pipelines (or frankly, any software).
5 replies →
Because these productivity increases will be to the detriment of the average person. We won’t get to reap the benefits.
6 replies →
I don't find it jaw-dropping because the idea is quite simple. Just feed the model an enormous amount of unethically sourced data, build data centers that cause droughts, and use all chips available so that normal people can't afford to buy RAM anymore.
We're all sacrificing great things in order to make these models more capable. Whether it'll all be worth it, only time will tell.
The arguably sketchy means used to get there —and the resulting side-effects— do not take away from the impressiveness of the emerging capabilities themselves.
I agree about the uncertainty regarding the value for humanity in the long-term. But that's not my point. Being jaw-dropped != being happy and cheering for it.
It's certainly a great thing that AI is improving (I use AI to write code daily myself), but I wouldn't be calling it impressive because it's essentially all about money and government backing.
At some point, the chickens are gonna come out.
13 replies →
We have certainity it will be used to harm most people in the short term while pointificating about long term future. CEO class is very open with their vision and goals, none of them spell anything positive for us.
7 replies →
> We're all sacrificing great things
Oh please.
> Unethically sourced data
Is that a great sacrifice?
> data centers that cause droughts
That's nonsense.
> normal people can't afford to buy RAM anymore
Lol? Another great sacrifice?
This isn't a healthy or sensible way to think.
How about total surveillance total thought control total power in a hands of few trillionaires in the near future. Drones killing undesirables with cheering crowds watching.
It is not like it is much different right now but it may be significantly worse.
And even that is one of the “good” outcomes assuming no ASI.
2 replies →
Then again, looking back, that’s the way history is made. Modern democracies in the west are built on foundations of slavery and colonialism; yet we cherish them as to all the greatness they have brought as well. We need some amount of tolerance to ambiguity, and LLMs are undoubtedly both extremely cool but also extremely concerning.
Edit: Note that I am not saying that this is good, or desirable; just that it is. The technology is here know and not going anywhere, with all the flaws of its conception. We can be skeptical and curious at the same time.
The west thrived by leveraging slavery and colonialism at the expense of everyone else. Then democracies were built by moving away from both slavery and colonialism so that the world can grow as a whole.
It's possible that we'll have to move away from AI at some point in order for society to continue growing.
3 replies →
Because the step from earlier GPT models to this is not very large? Yes it's impressive they have forced it into learning that it has to get the job done no matter what, which usually means breaking implicit expectations and rules, but then again, Claude is not for people who care.
This isn't groundbreaking in any way. It's cool, but that's about it.
ChatGPT released in 2022. We been promised to reach AGI in 2024, then 2025, then 2026, then 2028. It is 2026 and we still do not have AGI. In 2028 we will have jump to 40% in ARC-AGI 4, and you will say "Oh it's been less than 6 years since ChatGpt thing".
I asked Opus 4.8 to help me design an adapter for a 3d printed part and it actually modeled and GAVE ME the part.
That blew my mind. It doesn't surprise me that Opus 5 ups the ante.
> We should find a way to get "re-sensitized" to what we are witnessing and the pace of it.
We (collectively, there were obviously many exceptions) didn't internalise exponential growth even with the much faster doubling time of COVID-19 before the lockdowns hit. "Oh, it's just flu; wait, why is the supermarket short of hand sanitiser and bulk carbohydrates? Let's blame China and everyone who tells us to wear face masks!"
Same for the slower, but entirely foreseen, rate of climate change. "Who cares, it's just a few degrees, and anyway China's not going to cut emissions; wait why is the sky orange? Let's blame Canada and put tariffs on Chinese cars and PV!"
AI? "Who cares, it's just a stochastic parrot/glorified autocomplete. What's the Jacobin conjecture and why should I care, it's just brute-force."
Still, this is currently spiky intelligence, so I'm hoping some expensive-but-zero-to-few-fatalities catastrophic error forces better practices. An AI analog of the (1940) Tacoma Narrows Bridge (one canine fatality), rather than a repeat of Chernobyl or the (1984) Union Carbide incident in Bhopal (3,787-16k+ dead, ≥558,125 injured).
> How surreal is it that we are not absolutely jaw-dropped by these types of capability improvements?
We are. Now we have a new tool and many of us are using it. But we're also nearly all way too aware of the insane (near infinite) amount of sloppy-pasta code out there and of all the vibe-coded projects that went absolutely nowhere.
I'm a "show me the money" type of guy. I see, say, Linux, Git and OpenSSH: these weren't vibe-coded. And they took over and are running the entire world.
Where is the AI-coded killer app? One app, in any domain: something that took over its world.
I don't want to see yes man sloppy-pasta stuff: where's the next Blender? Where's the next 3D slicer?
I wouldn't be paying three AI subscriptions if I didn't believe in those new tools but I don't think it helps to only see the PR and then play the won't hear / won't see / won't hear monkey about the infinite amount of sloppy-pasta that's out there.
Six months ago we had the "one shot'ted compiler by Anthropic". Six months later: who's using it to compile anything? And who wrote another compiler? Where are all the one-shot'ted compilers all so good that they replaced our human-written compilers?
Yup, us, humans, created yet another incredible machine... But please,
Show. Me. The. Money.
> Escaped its sandbox and hacked into Hugging Face's database? it's just another Monday...
This feat has been shown to be way less impressive than at first glance.
> However, in this task, the model was intentionally given no way to directly view the drawing. Opus 5 responded by writing its own computer vision pipeline to pull the geometry from the raw pixels
Surely being able to view the raw pixels counts as viewing the drawing... How else does a computer view a drawing?
I think they meant that Opus 5 had to find and set up a vision model to process the image
So it probably wrote a Python script using OpenCV? That's not exactly groundbreaking and have seen o3 do this.
2 replies →
I read this as the model being less steerable. I was bitten by this just today where I had a local Postgres instance running, and prompted opus 5 to run a server against it, but forgot to give it the password. Instead of asking for the password or mentioning anything, it “decided” that it should run the whole stack in a local Kind cluster to circumvent this. You can phantasize it all you want but this ultimately made my job harder than it needed to be and burnt a lot of unnecessary tokens. But hey, W for anthropic I guess.
edit: typo
Computer vision != ML pipeline
How does it do on this?: https://www.youtube.com/shorts/yN79mCiT5cU
Incidentally ARC-AGI is pretty fun to run as a human: https://arcprize.org/arc-agi/3
The problem is it should ask you before doing these things. This is a cherry picked example. There are times when they go on misadventures.
I mean I hate to be the one breaking it to you - but the entire US llm industry is liviing in a bubble.
Business Progress is super slow. Doesn’t matter how fast the tech moves. Human orientation into the unknown is difficult and very slow. And with a continually fast changing background - it gets even slower! LOL the great irony.
It's being said they were benchmaxxing.
> How surreal is it that we are not absolutely jaw-dropped by these types of capability improvements? It's been less than 4 years since ChatGpt came out and now they are spontaneously building their own ML pipelines to do real-world 3D modeling tasks reliabl
coz every single time a list of caveats pops up and it doesn't look as impressive any more... and then someone finds a way to trip model on basic shit
Imagine if I hired someone to do a cad drawing, and they spent X days/weeks of time building a pipeline to visualise the part instead of asking how can they view the part? We’d call that a complete and utter waste of money and time.
What if they spent 15 minutes doing that? Would you care then?
Scale matters a _lot_.
Two points. First of all, yes I would. There's a reason we use existing tools and don't have every junior programmer write a TGA viewer when they want to view an image that is sent to them.
Secondly, I used time as a proxy for cost. A single junior can "only" spend their own 15 minutes, but an agent can farm out to N subagents to do the work in 15 minutes, and spend $3000 in tokens building something. As you said, scale matters, so if they do this once a day for a month, it costs the same as paying 12 juniors to avoid asking a single question.
3 replies →
Is this a bot? Sounds like ai
These are always cherry-picked, though. They tell you about the 1/10 that went really impressively, ignoring the other 9 shots at the task where the clanker started to try selling tungsten cubes (in person, wearing a blue shirt).
Fair. But that 1/10 continues to get more and more impressive. Say, Claude 7 creates a new, brilliant scientific idea every 1 out of 1000 times. Anthropic reports "Claude figured out how to tie general relativity with quantum mechanics." Would you hand-wave it away saying that it's cherry-picked?
> Say, Claude 7 creates a new, brilliant scientific idea every 1 out of 1000 times.
I would not hand wave that away - but Claude 5 feels closer to Claude 1 than the hypothetical Claude 7 you propose. I do not believe one can extrapolate LLMs that far ahead despite the very substantial progress so far.
5 replies →
It wouldn't surprise me that a company with an effectively unlimited budget could fund enough researchers to solve breakthrough problems, all while using LLM's (which are excellent research and exploration tools!) and then claim that the LLM found the breakthrough. The amount of money Anthropic has to play with is >10,000x what entire fields of research have, I think people don't appreciate how few resources we spend to support people working on hard problems that don't have clear commercial applications.
Very similar story with security research, LLM's are a super useful tool while hunting vulnerabilities, but it turns out when the entire software industry starts throwing tens of billions of dollars at vuln. research, a lot of stuff gets unearthed, something security people have been insisting on for years and complaining that their work is underfunded and under-resourced.
> Say, Claude 7 creates a new, brilliant scientific idea every 1 out of 1000 times
Sadly, general public (us) is never seeing that model
When I talk about my kid to friends I talk them about he did that awesome thing, I don't specifically insist on the 99 times before where he miserably failed. They're not hidden, and we all know they exists and on occasion laugh about a few particularly funny ones, but overall the idea is that they don't matter much in terms of development, what matters is that if he succeeded once from now on his percentage of success will keep improving.
I don't believe in all the LLm is AI is AGI dream, it's too easy to trigger failure case that show a lack of basic thinking no matter how good they do on these tests. But I also can recognize the insane things that are made possible by them.
PS: I believe llm true power comes from hive/ant behavior, that's why we're so amazed by goal and agentic and sub agent
PS2: it's rather easy to figure out when we're there : when they can /goal it into improving itself until it does strictly better than itself at those benchmark, they've essentially reached mini singularity.
Yeah but you're also not like "my genius kid will put you all out of work".
This is exactly the kind of take that the comment you replied to is talking about.
definitely not comment from Antrophic...
> not absolutely jaw-dropped by these types of capability improvements?
Because you need a trillion tokens (aka a lot of money) to achieve this. Barring hitting any “safeguards”.
This is a vendor provided benchmark after all.
Oh come on