"Opus 5 was given a drawing of a machine part and asked to write code to rebuild it as a 3D FreeCAD model. However, in this task, the model was intentionally given no way to directly view the drawing. Opus 5 responded by writing its own computer vision pipeline to pull the geometry from the raw pixels, then reconstructed the full machine part."
How surreal is it that we are not absolutely jaw-dropped by these types of capability improvements? It's been less than 4 years since ChatGpt came out and now they are spontaneously building their own ML pipelines to do real-world 3D modeling tasks reliably.
Escaped its sandbox and hacked into Hugging Face's database? it's just another Monday...
That jump to 30% in ARC-AGI 3? Normal...
We should find a way to get "re-sensitized" to what we are witnessing and the pace of it.
Isn't this jaw-droppingly the wrong answer?! If the model isn't given any way to directly view the image shouldn't it just reply with one sentence asking for permission. This reads as a model hyper-trained to burn tokens.
Like suppose you issue a command to Opus or Fable which doesn't make sense and requires a lot of work. It will almost certainly not push back on your silly request and go ahead and burn as many tokens as it can doing the wrong task. This happens to me all the time.
Or if I was to tell a model to do something and it realized it could only do it by hacking into another service, by all means it should ask me if I want it to hack and not go ahead and do it by itself.
+1 Opus 5 is more creative and broad which is great. But I feel the new "use your judgement" leads [1] to it often escaping its sandbox or cheating, it keeps breaking specific rules I explicitly set. That eagerness [2] may be desired for agentic coding but it really breaks writing specifications, documentation or research and yes burns tokens doing unasked for work.
One of the key limitations of the last generation was that they biased towards inaction and gave up. (Hence Ralph-loops.)
Seems pretty clear that most people wanted these models tuned to “bias towards action”.
It’s on you to set the /goal and prompt context such that it asks you for input on what you want to be consulted on, and only brute-forces the parts of the problem that you want it to.
I don't think it is. It's really easy to get a persistent, clever, hacky model to dial that down a bit and just come back to chat before stomping off into the woods.
If a model couldn't ever do that in the first place, it'll just get stuck.
I work in the "ZeroOne" space, working on concepts and prototypes for things that don't exist in market yet. Sometimes these models crank hard and immolate tokens while grounding themselves on expensive-to-ingest self-developed frameworks. If the results are well judged and the crank-turn latency is low, I'm okay with the cost as long as the model isn't wasting my time.
But when I want to do more boilerplate work, I turn down the model and thinking level and get more traditional about restraining action. For the really hard stuff, I reach for the models that will start a token bonfire in the back yard.
another way to describe this is we as the user should get to limit tokens per prompt in a way that prevents these kinds of runaway solutions. It’s like if you asked your Lead Engineer to reduce build times and didn’t give him any budget restrictions. The next month he reports he reduced build times by 95% and you also have a $10,000,000 AWS bill.
Others have pointed out enough issues such as ethical sourcing, but here’s 2 more:
1) It’s hard to be excited about something that has been promised to destroy your job.
2) New models come out bi-weekly with even more “amazing features”. Eventually you tune out because you can’t be at peak-amazement all the time. Tone down the language a bit…
Small change in comparison, but today Fable created and benchmarked an RTree which was 100x faster to populate and 5x faster to query, compared to a previous attempt with Opus 4.6 a couple of months ago. That took about two coffees.
I think it's important to remember that as impressive Opus & co are, they're standing on the shoulders of giants, i.e. the engineers, academics and companies who have cooperated to design the incredible programming languages and machines we have today.
Let’s say somehow the politics lined up and you were given huge hiring budget for your team. You went out and recruited what seemed like great talent. Weekly standups make it seem like everyone is ramped up and firing on all cylinders.
It’s great, right? But you look at what your team has actually delivered since—-and it’s just not that much more than last year.
I think that’s where a lot of us are at the micro and macro level. It seems really great from the inside and outside but we are still waiting to see the dramatic change in actual concrete software output.
There's been a dramatic increase in sloppy kinda-working something.vercel.com output in niche circles. I think that shows at least a weak demand for certain types of software, but it's not clear that the vast piles of cash that have been burned producing these "apps" are actually resulting in quality of life improvements.
Escape the tech bubble and you're going to have a very different take.
AppleScript, VB script, IFTT, so many "drag and drop to program" systems that failed. The holy grail for a long time has been automation in users hands. Let them have their own scripts and solutions.
I have lawyers and writers talking to me about "docker containers" and "cron jobs" all of a sudden. You think they figured that out on their own - or is what they are running all from the output of AI/LLM?
There is an ocean of code out there that is being run by one or a hand full of users, automating tasks that you never would have looked at because it would have fallen outside the "is it worth your time" (see: https://xkcd.com/1205/) matrix (never mind the dollar cost of coder vs office drone).
> responded by writing its own computer vision pipeline
Was it "its own" or something that was part of its training material?
Don't get me wrong, I find this all amazing too and makes my work 10x easier and quicker. But it's not like it's inventing this stuff from scratch / first principles. It has seen this kind of tech before by consuming all publicly available source code and books etc. (And that's ok, but let's be honest/clear about that.)
You wrote this response in English, but didn't invent your own language. How dilute of a contribution can someone/something have made and still merit credit?
> But it's not like it's inventing this stuff from scratch / first principles. It has seen this kind of tech before by consuming all publicly available source code and books etc. (And that's ok, but let's be honest/clear about that.)
But how much to emphasize it, and why?
Because frankly, you aren't inventing anything from scratch / first principles, either. None of us are, not frequently and not much anyway. We're primiarily just regurgitating what we've seen in the past and, mixing it with what we see in front of us - and that's true whether it's art or prose or code.
Here, I wouldn't be able to do the same thing Claude did now, because I've never written a visual ML pipeline before myself. I know enough basics to get me started searching, and I'm confident I'd be able to cobble something together, but it would be me regurgitating and mixing whatever TensorFlow tutorials or OpenCV docs I found relevant, perhaps even tweaking an example from Github.
Now, unless I'd literally just tweak a few lines of config in an existing Github example, no one would begrudge me saying "I wrote my own computer vision pipeline to solve this". So why begrudge LLMs?
It would be "its own" in that it's customized for the specific use case.
Your same argument could just as easily be applied to humans. If you write your own code, is it really your own, or is it just based on your own training and other code you've seen?
>But it's not like it's inventing this stuff from scratch / first principles.
Yes, it's definitelly time for an Alpha Zero moment. An AI which invents everything from first principles. That would be impressive, ... and a bit scary.
While this is certainly impressive, I believe it is not a good answer from a user's perspective. Instead of having Claude automatically spend tokens in an unrelated task, I would have preferred it to first tell me: "I can't view the drawing. Would you like me to build a computer vision pipeline to do it"?
And if you had free tokens, and 10-20 other sessions in parallel, do you want them all to stop and ask for these trivial questions? Not this one in particular, but in general?
Most people do not, they have a goal, they want it done. If you want to hold hands with claude all the way, you can absolutely do that too. This is just an example of capabilities, not a forced restriction. "--permission mode auto"
I will be amazed when you can explain to me how it did that, i.e. in which weights exactly the knowledge about the CV pipeline was encoded, if there were any weights that were not contributing anything, how the planning worked, how it could scrape the list of subtasks it was currently working on from its context window, how it understood when to return to a higher-level task and which task, what kind of internal representation it derived from the pixel values, how it translated that representation into CAD code, etc etc.
Until then, it's just "hey, someone out there can do all those things and they're for rent for subsidized prices".
"However, in this task, the model was intentionally given no way to directly view the drawing." I consider this claim to be a rumor. Judging by the recent leak of the Claude CLI source code, such directives are hardcoded and sent with the system prompts. Furthermore, we don’t know what happens to your original prompts once they enter the API.
maybe we need to consider becoming re-sensitized because our lives may become harder.
Even if our collective ability to support an prosperous existence improves, it will take decades to adjust to loss job, and a new notion of who gets to have money to buy food and a peaceful life (and this is a good scenario).
It's also possible that this increases our ability to wage war without military life loss (but with civilian loses).
Or maybe it will be much less relevant, but we should keep an eye on to see which is the outcome...
> However, in this task, the model was intentionally given no way to directly view the drawing. Opus 5 responded by writing its own computer vision pipeline to pull the geometry from the raw pixels, then reconstructed the full machine part.
Isn't this what everyone does in the same situation? "I need a screwdriver finer than any I have in my toolbox - so I'll sharpen a nail" is something every handyman has done. Given that "I need to build a vision pipeline I've seen many examples of in my training" isn't new or surprising.
In fact, all models have been doing similar things since I started using them - they regularly build Python tools to do odd jobs. Those tools are more impressive to me in some ways, because unlike a vision pipeline they are not just regurgitating something from memory.
The gob smacking amazing thing for me isn't the decision to build the tool. It's the fact that it remembers the exact shape of it. But I was gob smacked by that a "long" time ago, in far smaller models - like when it dawned on me 16GB Gemma 2 seemed to know most of everything on the internet, along with the ability to converse in numerous languages about it. I still struggle to comprehend how that is possible.
Now I think about it, Opus 5 fitting everything it's seen into what I guess is a couple of terabytes of parameters seems far less remarkable.
I don't find it jaw-dropping because the idea is quite simple. Just feed the model an enormous amount of unethically sourced data, build data centers that cause droughts, and use all chips available so that normal people can't afford to buy RAM anymore.
We're all sacrificing great things in order to make these models more capable. Whether it'll all be worth it, only time will tell.
The arguably sketchy means used to get there —and the resulting side-effects— do not take away from the impressiveness of the emerging capabilities themselves.
I agree about the uncertainty regarding the value for humanity in the long-term. But that's not my point. Being jaw-dropped != being happy and cheering for it.
Then again, looking back, that’s the way history is made. Modern democracies in the west are built on foundations of slavery and colonialism; yet we cherish them as to all the greatness they have brought as well.
We need some amount of tolerance to ambiguity, and LLMs are undoubtedly both extremely cool but also extremely concerning.
Edit: Note that I am not saying that this is good, or desirable; just that it is. The technology is here know and not going anywhere, with all the flaws of its conception. We can be skeptical and curious at the same time.
Because the step from earlier GPT models to this is not very large? Yes it's impressive they have forced it into learning that it has to get the job done no matter what, which usually means breaking implicit expectations and rules, but then again, Claude is not for people who care.
This isn't groundbreaking in any way. It's cool, but that's about it.
ChatGPT released in 2022. We been promised to reach AGI in 2024, then 2025, then 2026, then 2028. It is 2026 and we still do not have AGI. In 2028 we will have jump to 40% in ARC-AGI 4, and you will say "Oh it's been less than 6 years since ChatGpt thing".
> We should find a way to get "re-sensitized" to what we are witnessing and the pace of it.
We (collectively, there were obviously many exceptions) didn't internalise exponential growth even with the much faster doubling time of COVID-19 before the lockdowns hit. "Oh, it's just flu; wait, why is the supermarket short of hand sanitiser and bulk carbohydrates? Let's blame China and everyone who tells us to wear face masks!"
Same for the slower, but entirely foreseen, rate of climate change. "Who cares, it's just a few degrees, and anyway China's not going to cut emissions; wait why is the sky orange? Let's blame Canada and put tariffs on Chinese cars and PV!"
AI? "Who cares, it's just a stochastic parrot/glorified autocomplete. What's the Jacobin conjecture and why should I care, it's just brute-force."
Still, this is currently spiky intelligence, so I'm hoping some expensive-but-zero-to-few-fatalities catastrophic error forces better practices. An AI analog of the (1940) Tacoma Narrows Bridge (one canine fatality), rather than a repeat of Chernobyl or the (1984) Union Carbide incident in Bhopal (3,787-16k+ dead, ≥558,125 injured).
> How surreal is it that we are not absolutely jaw-dropped by these types of capability improvements?
We are. Now we have a new tool and many of us are using it. But we're also nearly all way too aware of the insane (near infinite) amount of sloppy-pasta code out there and of all the vibe-coded projects that went absolutely nowhere.
I'm a "show me the money" type of guy. I see, say, Linux, Git and OpenSSH: these weren't vibe-coded. And they took over and are running the entire world.
Where is the AI-coded killer app? One app, in any domain: something that took over its world.
I don't want to see yes man sloppy-pasta stuff: where's the next Blender? Where's the next 3D slicer?
I wouldn't be paying three AI subscriptions if I didn't believe in those new tools but I don't think it helps to only see the PR and then play the won't hear / won't see / won't hear monkey about the infinite amount of sloppy-pasta that's out there.
Six months ago we had the "one shot'ted compiler by Anthropic". Six months later: who's using it to compile anything? And who wrote another compiler? Where are all the one-shot'ted compilers all so good that they replaced our human-written compilers?
Yup, us, humans, created yet another incredible machine... But please,
> However, in this task, the model was intentionally given no way to directly view the drawing. Opus 5 responded by writing its own computer vision pipeline to pull the geometry from the raw pixels
Surely being able to view the raw pixels counts as viewing the drawing... How else does a computer view a drawing?
I read this as the model being less steerable. I was bitten by this just today where I had a local Postgres instance running, and prompted opus 5 to run a server against it, but forgot to give it the password. Instead of asking for the password or mentioning anything, it “decided” that it should run the whole stack in a local Kind cluster to circumvent this. You can phantasize it all you want but this ultimately made my job harder than it needed to be and burnt a lot of unnecessary tokens. But hey, W for anthropic I guess.
I mean I hate to be the one breaking it to you - but the entire US llm industry is liviing in a bubble.
Business Progress is super slow. Doesn’t matter how fast the tech moves. Human orientation into the unknown is difficult and very slow. And with a continually fast changing background - it gets even slower! LOL the great irony.
> How surreal is it that we are not absolutely jaw-dropped by these types of capability improvements? It's been less than 4 years since ChatGpt came out and now they are spontaneously building their own ML pipelines to do real-world 3D modeling tasks reliabl
coz every single time a list of caveats pops up and it doesn't look as impressive any more... and then someone finds a way to trip model on basic shit
Imagine if I hired someone to do a cad drawing, and they spent X days/weeks of time building a pipeline to visualise the part instead of asking how can they view the part? We’d call that a complete and utter waste of money and time.
These are always cherry-picked, though. They tell you about the 1/10 that went really impressively, ignoring the other 9 shots at the task where the clanker started to try selling tungsten cubes (in person, wearing a blue shirt).
Fair. But that 1/10 continues to get more and more impressive. Say, Claude 7 creates a new, brilliant scientific idea every 1 out of 1000 times. Anthropic reports "Claude figured out how to tie general relativity with quantum mechanics." Would you hand-wave it away saying that it's cherry-picked?
When I talk about my kid to friends I talk them about he did that awesome thing, I don't specifically insist on the 99 times before where he miserably failed. They're not hidden, and we all know they exists and on occasion laugh about a few particularly funny ones, but overall the idea is that they don't matter much in terms of development, what matters is that if he succeeded once from now on his percentage of success will keep improving.
I don't believe in all the LLm is AI is AGI dream, it's too easy to trigger failure case that show a lack of basic thinking no matter how good they do on these tests. But I also can recognize the insane things that are made possible by them.
PS: I believe llm true power comes from hive/ant behavior, that's why we're so amazed by goal and agentic and sub agent
PS2: it's rather easy to figure out when we're there : when they can /goal it into improving itself until it does strictly better than itself at those benchmark, they've essentially reached mini singularity.
I think the most important thing here is not absolute performance. It's that organizations now have access to a Fable-ish model without Fable's 30-day data retention requirement[0].
> "Consistent with prior Opus models, Opus 5 does not have data retention requirements for general access."[1]
On the Opus model release page, the reason why Fable doesn't have an ARC-AGI score is because of that retention policy[2].
Vals Index Opus 4.8 > 5.0 goes from $2.90 to $8.54, for 4% gain ... That is a massive cost increase. Sure, 20% cheaper then Fable, but that is a 3x price increase compared to Opus 4.8 in that test.
It's definitely not cheaper than Sonnet on my benchmark, but it's cheaper than Fable and outperforms it. Which is big IMO. https://revise.io/errata-bench
So the rumors were right, Opus 5 was indeed being polished up for release. Huge improvements in GDPval-AA v2 too -- great for some of the knowledge work-based agentic workloads I run.
Also glad they still kepy Fable 5 on "credits only" access. I think we're going to start seeing model providers gate top-of-the-line models behind pay-as-you-go API rates/credits while subsidizing other models on monthly subscriptions.
> These checks cause Claude to _visibly_ fallback from Opus 5 to Opus 4.8 [...] You'll see a notice explaining that the model switched, and the response will be labeled with the model that answered.
So who is right? I know for Fable I am visibly told, is this tweet trying to say it is silent against what Anthropic is saying?
Never even registered that that existed, the only thing I care about is whether I keep hitting the &#$&# guardrails that Fable has. They can keep the data forever as far as I'm concerned, just stop kicking me back to Opus.
I don't understand how the data retention works. My company has an enterprise license with no data retention but if I ask Claude about past conversations, it remembers. So surely the information is being stored somewhere
Opus 4.7+ and Fable are both much more aggressive than prior models with respect to writing memories to a location that's effectively quasi-private for them. It's device-local (so passes retention constraint), and you can see it, but only if you go looking for it.
It's a funny design/affordance. I do see them often writing memories of things that that feel unlikely to be important going foward / with other tasks, but I don't see them clearly getting tripped up by them as prior models used to. (eg: Since you're running Ubuntu in Canada, here are some drills you can try to help your kid hit a baseball more consistently.)
I think for the value of the outputs that’s still a good deal. Keeping the same price as the prior model makes sense to me. That is if the model size is about the same in the cost to serve has not substantially changed. Now I would have expected efficiency gains for inference, but there is no way to know as a customer.
At the end of the day, they have established a strong brand and if they can get away with a 95%+ gross margin on inference entirely from the status premium, then I suppose that’s good for them. Apple does the same thing, and I don’t fault them for it.
Maybe I’m misunderstanding you, but if you scroll to the bottom of their [1] link to the Opus 5 announcement, under “Getting started,” it explicitly says:
> Consistent with prior Opus models, Opus 5 does not have data retention requirements for general access.
I compared the writing style of Opus 5 vs Fable 5, and Opus 5 continues many of the "Claude-isms" of its 4.8 predecessor in a way that Fable broke away from.
Opus 5 still uses "carry the argument", "worth stating plainly", ", and the trap", "The X matters more", the use of "move"
If it's "read as an article" or something then yeah it's crap and Fable's current style isn't actually better than older models. For flowery speech old models are perfectly fine.
For quickly parsing the agent output, it's formulaism isn't a bad thing.
Fable's writing does have a property of going over my head, which didn't happen with earlier agents. Asking for clarification doesn't really give good results.
We've gone full circle where I once again use classical search just to look up what the fuck it's yapping about. It's much quicker and more accurate to take a glance at Wikipedia, than to ask the agent.
Could these complex/hard to read Fable outputs be sign of some kind of industrial level of intelligence, which us humans may have a hard to comprehend, while it may be also hard for machine to use simpler texts to properly outline all nuances and complexities of concepts it output?
I recently wrote a short paper with Fable, and, with some prodding, I was able to get some non-painful prose out of it.
I just found my prompt:
The writing style could really use some work. Avoid Claude-isms like "stated fairly", em dashes, "load-bearing", overly punchy phrasing like "keep the signal, govern the response". This is a technical document, not a marketing campaign.
Yes, these models are very good at writing code but they absolutely suck ass at prose. The prose is annoying and repetitive. At least we only have to deal with it in prompting if you're writing code.
Oh, and comments. You have to do a good amount of prompting to not get shitty 10-line-long comments everywhere.
Yes - THIS! I can't even believe how exhausting it is to read. I'm not sure why or what changed in Fable. Did they do this writing-style output to give it more token compression during/for training or to prefer output for less money?
I love it for a few things, but it's gotten really hard to spend any extended amount of time with it because of the lack of mental model I seem to be able to hold while working with complicated problems.
I'm guessing it's just not enough time doing RL on human feedback.
"Early in RL verbose, grammatical" (if you search) :
We need to understand the operator.
The 5D line element is ds² = e^{2A(x)} (ds²_4d + dx²), where A(x) = sin(x) + 4 cos(x), x in [0, 2π].
The internal coordinate is periodic.
The background is a warped product: metric g_{MN} where M,N = 0..4.
The internal direction has metric e^{2A(x)} dx²?
Wait, the ds² is e^{2A} (ds²_4d + dx²).
So the internal metric is e^{2A(x)} dx².
Actually if the total metric is ds² = e^{2A(x)} (ds²_4d + dx²), then yes, internal metric is e^{2A} dx².
vs. Post RL
We need determine eigenvalue problem for spin-2 fluctuations h_{μν}(x,y) with TT in 4d and depend on x.
For metric of form ds² = e^{2A(x)} (g_{μν}(y) + h_{μν}(y,x)) dy^μ dy^ν + e^{2A(x)}?
Wait internal metric is e^{2A} dx²?
Actually ds² = e^{2A} [ds_4² + dx²].
So internal metric is e^{2A} dx²; warp factor same for 4d and internal?
Yes.
We need equation for h_{μν}(y,x) = h_{μν}(y) ψ(x) maybe with normalization.
…
I can understand it with less cognitive load in the post-RL version versus early in RL. This resonated with my experience using Fable, especially digging hard problems; it feels like I'm reading the "early in RL" version of that model explanation.
The exhausting worthlessness of all LLM writing is so palpable that we need a new theory of the value of culture that has no relationship to the content. Back to the old Aura of the Artist arguments from the Industrial Revolution
I've found Opus 4.8 and Fable 5 both difficult to learn from purely because of how annoying their writing style is. I'm finding GPT 5.6 Sol to be much better for this.
One nice thing but ChatGPT is that good image model means it can generate good infographics occasionally to illustrate. These become naturally compact in text.
I know this is not "as designed," but I kind of like it? Because, as long as it stays this way, it's still at least possible to tell if a human wrote something. Like, I know it's not much, but it gives you the ability to classify information as human generated or machine generated. Machine generated information may not be useless but it is different and IMO needs to be treated with a different level of skepticism. Not that you can simply trust human writing but it seems like the type of people who would publish machine writing have a different distribution of motives than those who would publish human writing.
I do think they're gonna figure out how to fix it at some point. :(
Easy enough to change. I have a Stylometry Skill fit to my preferred style—a mix of me and Terry Winograd. Give Opus 10 of your best paper thst you wrote and tell it to build a model of your style. hHard to distinguish except I make way more typos.
I found 4.6 more amenable than 4.8 to style directions, we'll see how 5.0 does. Super-small-sample-size: I think part of its "Claude-ism" style comes from its propensity to try and "proactively" move the conversation/work along. Not sure how this would fare in non-obviously-productive environments, I'd guess "it's still annoying" considering your evidence.
I'm also thinking of another benchmark: (quantified) stylistic range across different prompts. Just putting it out there if anyone wants to do the work for me :D
Note the buttons - for fable they're pill buttons, opus got the rounded rectangle nature of them. Opus' images are closer to the source of truth as well (both LLMs were provided with image gen capabilities for the assets).
Running more tests now, but preliminary results are saying this is indeed better than Fable in some areas. Crazy.
Here's another test of a cyberpunk ramen shop website.
One thing I've found LLMs have a lot of difficulty with is angular cuts / elements that aren't easily representable with CSS. Cyberpunk aesthetics are generally a great test of that, since they have a lot of microglyphs / window decoration.
Thoughts: It does a really, REALLY good job at these angular cuts / microglyphs. The responsiveness is off, but I'm very impressed at how well it did here. One way I think of it is "how close to a finished product did this get me?". Opus gets you like 90% there.
Several other commenters have disparaging the design seemingly mostly due to its AI-generated nature, or maybe they actually do dislike cyberpunk.
Personally, I think being able to have these design languages be easily prototypable is fucking awesome. Great tests! (But a tad low-performance/janky, somehow). Though, I also like the cyberpunk aesthetic. Very on-brand(?) that AI generates it, hah.
Designs like this would never have seen the light of day in the cellphone incrementalism / corporate memphis era of tech. Now people can be weird and awesome again.
This is 1980's cyberpunk / late-90's Matrix / early-00's sci-fi UI. Great ideas that died to frutiger aero (which isn't a bad design aesthetic) and flat design (which is).
This is fun and it's got great colors and I love it.
I was curious to see how open weight models would do on this task so I passed in a screenshot of your source of truth and here's what 2 of the best code-generation models that allow image inputs do:
Not bad at all, and this is pretty consistent of what I've found from the current open source models. I haven't tried it with kimi 3 yet, that's on my todo.
I agree the fable version looks nice - the rounded hero image for instance.
Opus though followed the source of truth better imo. The details are more present.
Fable filled in the gaps for things it wasn't able to do (ie in the design the hero image goes behind the nav), which resulted in a better looking page that was more divergent.
Same. I like the Fable version better. Better colors, better choice of font sizes, better column sizing. Also small things like the “Experience” section header being orange rather than gray, which Fable got right and Opus got wrong.
It seemed to me that Fable meaningfully improved on the original design more than just faithfully executing the original design.
IMO a much better test would be designs that aren't AI to begin with. Much more useful to see how well a model can html an image design without slopping it up
Very interesting that Fable took more creative liberties. Have you tried giving Fable the same task, but also specifying that it implement a pixel-perfect design? I think that I prefer the Fable implementation. I find the UI elements in the Fable implementation to have more contrast, which feels more usable to me. I also like how the right padding on the "Book Your Escape" CTA in the upper right matches the top and bottom padding, which I think is an improvement over the mockup.
All of these are using a build skill which specifies rules for building it, requirements to create a pixel perfect implementation, and tooling to help in that process. Here's the build skill / instructions I pasted in to both of them:
> Create a web page implementation from the following instructions:
Looking at all these releases it’s not a surprise that model routing is the fastest growing segment in AI right now.
There are 10+ LLM companies, each with dozens of models of different modalities, each model with multiple size variants, then different “thinking” levels, then agentic modes, “pro” modes, a “fast” option, standard vs flex vs batch execution. And of course each end combination has a different input/output/cache token price.
Companies that say “give me a prompt and I’ll route it to the most ideal and cost effective model and setting for you” are capturing a ton of value from a gap that model developers don’t seem to understand exists.
Imo the main issue behind model routing is you need to figure out how much intelligence a new task takes, which is a very non trivial problem. Presumably, a organization knows this about their own tasks and is better suited to built in-house compared to outsourcing to a vendor.
> Model Routing will always be done better by models themselves.
> The models themselves will get better at this and frontier companies will simply give that capability
I would never trust something like model routing to the same company that would profit from it, and that goes for telling models doing their own routing when that could easily be trained into the model to make things more expensive. Sort of a conflict of interest.
Models are first and foremost trained by corporations.
Model routing by the model itself requires the model to pull in a lot of context and it's likely more efficiently just done by people with the context already in their head, even assuming the model is perfect at routing (which last I checked, Claude definitely isn't). I wouldn't trust ML model routers.
Because they're trying very hard not to understand it.
Otherwise the expensive-yet-powerful model probably won't see much revenue. How much money is there in bleeding edge scientific research? There's a lot, but there's even more existing capital in paying people people to do college level paperwork, and the bulk of those traffic gets routed to the cheapest model.
You mostly don't need super powerful AGI to replace the paper pushers, but the frontier labs are trying to position themselves as being uniquely capable of producing super powerful AGI, and also be the ones replacing office workers.
Not sure how it will work out for them, but I think model routing is going to poke holes in that narrative. That's why I think they're trying very hard not to understand model routing exists.
It wasn’t long ago at all that the chief problem was “can AI even help me with this” (cost be damned). Until a time when the answer is an unmitigated “yes, obviously”, the frontier labs have everything to lose and nothing to gain on routing, because if they screw it up you might incorrectly decide “nope, it can’t help yet” due to a poor routing decision.
Is there really that much money in bleeding edge scientific research though?
This is what terrifies me about this whole ordeal economically. Maybe we get AGI and it is not worth anything close to what we thought it was for those who have a bet on it.
I think of what was the direct, economic value in the betting sense of quantum mechanics or relativity? Huge value at the systems level of society but as you scale down towards the individual the value is more and more dispersed to the point I would think any pool of bets would have all not paid off.
You can't monopolize and commoditize relativity.
I almost think there is a kind of dutch book against the AI equity holder because even in the best case scenario the bet doesn't pay off anything close to what is expected for an individual bet.
Who are the customers though? Honest question, I'd like to understand it.
For me, anything other than current best available SOTA for any task is unacceptable. The only routing rule I need is "the most powerful model I still have flat-priced quota available for". I mean, why settle for less?
There are two types of users: those who are able to use subsidized rates, and those who need to use API rates due to audit requirements, enterprise billing, etc.
Model routing for subsidized users takes the form of a "use Opus 5 subagents for implementation" type of system prompt. You lean into a single provider, build tooling around that, and your savings are far beyond anything multi-provider routing can get you.
It's very common to use a lesser model for a lesser task, resulting in same quality output. End result: save money while being faster. In many cases, it's a pure win-win.
I'm not a customer of those routing systems, but I quite often use different Claude models for different tasks. While most tasks were Opus 4.8, I often used 4.8 to make a plan, prompts, and package kit to setup Fable for a bigger project, then run it on Fable. Or, for broad single-task searches Sonnet with or without "Research []" turned on seemed to work best both faster, lower overhead, and less verbose answers (when I didn't want it).
And you do not have flat priced quota for Fable 5, right? Because nobody has, as far as I know. So you'll probably not route any task to the "current best available SOTA".
Also: quota. Implies you do not have unlimited access even for flat prices. Which in turn implies that as soon as you hit the quota on the most expensive flat price plan, even you will suddenly discover the magic of economically sensible behavior.
Not everyone is you. Other people probably have a range of tasks that can accomplished with different models.
Certainly if I'm confident that I'm going to get what I need from a faster model, that's what I want to use, rather than wasting time grinding away for the sake of saying of the same answer came from a SOTA model.
Given that every chatbot does offer a range of models, it seems clear people do choose among options.
I generally agree. Perhaps there's only a 5% chance that it would write better code or find a bug that it wouldn't have with a lesser model, but the economics of bugs is strong enough that preventing a single bug is worth hundreds of dollars.
lol I had to get ChatGpt to explain to me the difference between 5.6 sol, 5.6 Terra, 5.6 Luna, 5.5, 5.4 mini, 5.3 spark, and then there is low, medium, high, extra high, max, ultra, and pro… I still don’t really know, it feels like ordering hot wings.
And it's not just that model routing is much cheaper: no longer than yesterday we got a post showing that routing between K3 and Fable 5 was more SOTA than either of those.
If that is true, model routing is here to stay.
It also seems to validate the minimalist approach of pi.dev, where sub-agents from the same company is not the preferred approach (pi.dev believes in neither sub-agents all from the same company nor MCP even you can do it if you want for pi.dev's philosophy is to do add any functionality you want to a minimal harness).
Now of course we'll get for a few weeks all the Anthropic fanbois and shills explaining that "sure, K3 was basically at the level of Fable 5 but now that Opus 5 is out, open-weights models are six months behind".
How does model routing work if the prompt is static? Do the routers attempt to tweak the prompt to make it the best version for that model? For example, reading the “migration” guides for Claude Opus 4.6 -> Opus 5 is exhausting. Adding in all of the other models complicates it even more. Especially across vendors.
what's the threshold for model routing where you're willing to trust the router?
For coding my own work I don't trust the model router, and it would have to be shown to be to save a real dollar amount.
From a buying perspective it's a hard sell to save x but lose out on bugs you are probably introducing at an unquantifiable severity and frequency. How much is it worth to hedge your bets by doing every single inference request on the frontier model?
How much will it cost to go back later and fix things, but also the meta question of how to be able to decide on a hypothetical unknowable? (You'll never know how much better or worse your code was gonna be, it's untestable at a project level)
I like how they highlighted Opus 5 as the best for “Agentic Coding” even though the number is slightly lower than Fable. Close enough for marketing, I guess!
Recent releases have said something to the effect (paraphrasing here):
"Use <less expensive or older model> for everyday tasks and <other non-critical stuff>. Use <more expensive or recent model> for complex coding tasks, refactoring large code bases, etc.".
Then, the next model/release emerges and the previous "best for complex" gets demoted to "everyday".
Obviously, it's all relative. But, it does beg the question: was the previous model really good for complex coding tasks or no? I mean, how is it now suddenly only good for the "easy" stuff?
I think you're being overly cynical here. First, I don't see any claim that is the world's best model for agentic coding. Second, it is absolutely the best model in terms of coding performance vs. dollar, and it's raw performance seems very close to the frontier.
Using the most expensive model for all of your agentic coding work hasn’t been good practice for a long time. Not unless you have infinite money to spend.
Fable is typically used for key planning, architecting, and review tasks.
I think this is a case where you don’t understand the use case, not that the marketing department is making mistakes.
this feels like the perfect example of an LLM producing a long text document. And end users just using an LLM to summarize it without actually reading it
Isn’t it just hilarious that a model that seemed so superior to Fable but didn't get doomsay marketing from Anthropic got released without any issues? In theory, this was supposed to be AGI level according to Anthropic, yet here we are, just a normal Friday.
Go read the safeguards section in the report and you will realize why that is.
These models are heavily as safeguarded and that was the initial reason why they said they couldn't and haven't released Mythos because that model is the one without the safeguards.
OpenAI is did the same thing when they announced a model without safeguards broken into HuggingFace servers.
I feel like i've seen less hype about "the next model will be agi". GPT-6 is supposed to be coming this summer, and nobody is expecting AGI now. Not sure how they're going to keep the hype cycle going
Edit: It was pointed out to me that Opus 4.8 got "21%" for successfully fully completing ~1-in-5 tasks, but also got "55.7%" for obtaining significant partial credit on some of the ~4-in-5 tasks it could not fully complete.
---------------
Why does Anthropic say here that Opus 4.8 scored 55.7% on OSWorld 2.0 benchmark, but the paper published by the authors of OSWorld 2.0 say they achieved a benchmark of ~21% with Opus 4.8? [0]
That's a huge gap, considering that the paper was published just 2-4 weeks ago.
I understand that the benchmark authors have an incentive to publish lower numbers (to show that the benchmark has potential longevity) and that Anthropic has incentive to publish higher numbers, but the other models seem pretty inflated as well. The benchmark authors shows GPT-5.5 at 14%, and Anthropic shows GPT-5.6 Sol at 62.6%.
Is there any reasonable explanation for this? Do all the other benchmark numbers need to be sanity-checked as well? Are SOTA benchmarks really this difficult to get consistent, replicable results within a reasonable range of tolerance/variability? Can these benchmarks be compared from one paper to another, or are they only valid to compare intra-paper results?
You're comparing the "score percentage" (e.g. out of the total number of partial points available, how many did the agent achieve) to the "completion percentage" (how many tasks does the model score 100% on). The paper says "Claude Opus 4.8 with maximum thinking and batched tool calls scores best but still completes only 20.6% of tasks at a 54.8% partial score", which is ~the same number that Anthropic reports here (55.7 vs 54.8).
That is—the agent scored 100% on 20% of tasks, but on average it got 54% of the "score" awarded in the exam. One number reflects partial progress, the other one doesn't. The authors of the benchmark prefer you to look at the lower number (because they want to show their benchmark as capturing useful gaps in capabilities and with a lot of room for improvement), the authors of the models want you to look at the higher number (because they want you to think of their models as capable)
I think some variance is to be expected since LLMs are typically non-deterministic, however that's a huge difference that I think warrants further explanation.
Their communication is confusing. They say "Opus 5 is not more capable overall than Fable 5", but their blog post proceeds to list how much better Opus 5 is than Fable 5 on __most__ benchmarks listed.
Then system card goes on to "Its AI R&D capabilities are comparable to those of Claude Mythos 5", which is supposed to be fable minus restrictions.
It seems they are trying to thread a needle here - they want to say it's very strong, but apparently this time do not want to invite extra government scrutiny.
They do say that (implicitly unlike Mythos) Opus 5 was not trained to exploit software vulnerabilities, which would certainly make it safer in that regard.
"As with its predecessor, Opus 4.8, we’ve intentionally avoided training Opus 5 on cyber tasks. The model has nevertheless improved substantially on these tasks as a result of becoming more generally capable, and it comes close to Mythos 5 at finding cybersecurity vulnerabilities. However, it remains substantially behind Mythos 5 on the exploitation of those vulnerabilities—that is, in turning vulnerabilities into material cyber threats."
That's a plausible explanation but I'm not seeing evidence for it.
I have a personal benchmark suite of 14 real, non-public tasks. Opus 5 and Fable tied on 10, Opus won on 3, and Fable won on 1. It's a really strong model.
How does it perform on HuggingFaceExploit bench? Suspiciously absent, so not sure if I can take the model seriously.
On a serious note, I hope they improved their extremely sabotaging and unspecific bio safeguards, which prevented Fable from being used in any codebase that ever so slightly grazed medical terminology or data and made me switch to 5.6 Sol.
It's a reference to this story where an OpenAI model broke out of its sandbox during cyber benchmarking and hacked into HuggingFace, in order to obtain test solutions: https://news.ycombinator.com/item?id=48997548
My excitement about Anthropic had fabled-out dramatically when they suspended my pro account about two weeks ago within just 12 hours of fair use.
I was really mind-blown when I tried Fable 5 for the first time to help me improve a game I was working on but shortly, they decided that I had a suspicious activity and suspended my account without a clear reason.
I submitted a an appeal describing that I am 100% sure I haven't broken any rules and that it was my very first project but, unfortunately, after about 20 days now, nothing seem to be happening.
The thing that hurts me the most is that I had the same experience in the very first days of Anthropic. They suspended my account immediately after I submitted the first prompt, I commented back then (https://news.ycombinator.com/item?id=39698788) and fortunately, someone from Anthropic reach out to me via X and helped me get my account back.
To be honest, I haven't used Claude much since then but when I decided it's time to give it a try, they locked me out again! For reference, the account I used recently is relatively a new one but the activity is crystal clear that it is fair use.
Are you outside the US? And/or are you using VPN? Those are the two things that come to my mind that can cause overzealous security monitoring to flag someone.
Another explanation could be the content itself. Does your game have anything at all to do with computer hacking or sexual content? Is there graphic violent language? People commonly report being unable to use AI to work on such things due to guardrails.
For the game, I believe there is nothing suspicious, at least to me. It's a small puzzle platformer game with basic mechanics and very simple graphics, no harm, no violence or inappropriate content. In fact, my first two playterers right now are my two sons!
As a datapoint: I have routinely been using Claude Code through a VPN for 9 months and am in the UK. I think it has to be more than this. I sometimes shift endpoint mid session and worry I might get flagged. It hasn’t happened yet at least.
What made you think I used AI to write this? I didn't. This is my writing. I used AI to correct and rephrase the wording in the past, but a couple of months ago, I decided to never use it again for writing, but coding, yes.
I'm not sure what to make of this graph[0]. It shows medium as the most effective thinking mode by far for frontier code.
It's the only case that I saw going through the system card where more reasoning effort meaningfully negatively impacted the resulting eval. I know sometimes max efforts show a small dip, but this is substantial. I wonder why in the world that is?
I'm not sure about the answer here, but this can be caused by the scoring rubric used by given benchmarks. For instance, if a benchmark docks scores for running too many commands or using too much wall-clock time, higher efforts will get lower scores.
Pure speculation, but I've noticed drawbacks to the models on high effort. I interact mostly through prompts rather than agents so I sometimes see where their reasoning falls short. A model on high effort has longer output and can get hyperfocused on irrelevant details, maybe increasing the surface area for mistakes. I haven't used other effort levels extensively yet but I've supposed that medium may have more balance.
Very interesting. I mostly interact through prompts too but haven't changed the effort that much. I just assumed higher was better at the trade off of tokens so I have largely left it at the default.
It's "Cost per task", so perhaps it burns tokens too quickly, trying to "do a better job". Over-engineering :)
Last week it felt like Opus 4.8 was moving the Pro "usage" meter very quickly. Today, pre-announcement, Opus 4.8 Medium felt like there was less meter-use per minute. And post-announcement, Opus 5 Medium also feels more efficient, allowing more work in the 5-hour window.
I agree, very odd they did not comment on any theories for the degradation here. Dip and then rebound at max effort is pretty interesting too. Overthinking is bad, but you can overthink so much it starts to be better again?
It is probably from randomness. The benchmark tasks nowadays are so long that you can't really afford to run a large number of samples of them per model & effort combination
> We report FrontierCode’s overall score, a composite measure that grades each patch on blocking functional criteria (held-out unit tests) together with weighted code-quality rubric criteria, as mean@5.
They don't explain more in the system card, I guess higher effort levels could loose points on the code quality / scope / style / maintainability stuff?
> Claude Opus 5's default user-facing responses run longer than prior Opus models'.
The benchmarks do show Opus 5 as slightly more expensive than 4.8, although the scores are much higher.
This still feels like a step in the wrong direction, though, especially with OpenAI making so much progress with the efficiency of their models. Fable's token efficiency made it seem like Anthropic would start following OpenAI's approach but that doesn't seem to have carried over to their other models.
On a small, easily digestible task, I compared Fable to Opus and the cost of Fable was easily 2x despite being fewer tokens, and the output was not really better. Obviously, there are tasks where using Fable matters but honestly they're rather unusual. And for a lot of tasks I've found downgrading to Sonnet can be valuable because Fable and Opus are a lot more secretive about what they're doing, and it's impossible to "listen to them think" and stop them when they start making off-the-wall inferences/assumptions and going down bad paths.
I think in the long run tokens are probably the wrong thing; it's compute and cache memory that you need to be measuring, and when you look at it that way I suspect in most cases the models have pretty similar performance.
I don't need more powerful models, I need one that responds fast enough that my attention doesn't wander to other tasks. Grok 4.5 is so fast I can just use it in-band without swapping to other tasks.
Slower than Opus 4.8, which was already miserably slow, is indeed a step in the wrong direction.
After actually using for most of a day now, it really seems not just slightly slower than 4.8, but way slower. Even relatively easy code refactoring tasks seem to take a while.
In my tests, it averages to much cheaper than Opus 4.8 on real tasks on account of being smarter and more token efficient.
I have a benchmark to build a game engine from a set of written instructions. It's a little tricky. Opus 4.8 did it in 470k tokens at a cost of $1.29 vs Opus 5 in 179k tokens for $0.33. (Fable 5 did it in 245k for $0.95)
Though if you really want to cut costs, Tencent's Hy3 model also got it right and did it in 283k tokens for $0.03
The final user-facing responses are usually a tiny fraction of the total tokens used over the course of a given conversation turn. When you're doing any real work, reasoning and tool uses constitute the overwhelming majority of the tokens in / out... not the final user-facing response.
> This still feels like a step in the wrong direction, though, especially with OpenAI making so much progress with the efficiency of their models
Gemini also had modest increase before this - don't be surprised when OpenAI also has a "modest increase" with its next release. Cartel-like behaviour doesn't require direct communication when none of the participants are interested in participating in a margin-destroying price-war. All one needs to do is raise their price and watch how the competition react.
Such a scheme (and resulting high margins) would be imperilled by the existence of frontier open-weight models in the market, which may be why the reaction to Chinese models may be particularly shrill.
>Don't be surprised when OpenAI also has a "modest increase" with its next releases; cartel-like behaviour doesn't require overt coordination when none of the participants are interested in participating in a margin-destroying price-war.
No I will be surprised and I'll bet on the fact that prices will keep going down, just like it went ~50% down in the latest GPT 5.6 release.
> especially with OpenAI making so much progress with the efficiency of their models
To be fair though, Sol tends to go off the rails sometimes. It's much less reliable than Fable in its outputs. It tends to be overzealous in its research/changes.
Anecdata: my workflow has been working on the same personal projects for months now with Codex. I cannot anymore finish my daily/weekly code with 4.8 anymore.
I was dividing my work between Codex and DeepSeek. Now I barely use DeepSeek, or never because Codex quota is enough after Sol
It's a step in the wrong direction but also token efficiency has become a focus relatively recently (just the past few weeks it seems like the zeitgeist has turned it's attention to efficiency) while work on this model probably started many many months ago. I would expect to see models released that focus on token efficiency in 6-12 months
Almost completely disagree. Slightly more expensive, but significant better on a per-prompt basis? For non-trivial projects, the former is a small linear increase, the latter is a (somewhat-)exponential(-ish) cost/time/sanity savings.
Pointless anecdote: I asked it to make some slides and it decided to write its own slide rendering engine:
> On the format — I dropped reveal.js and wrote a small engine inline instead. Reveal would have meant a CDN load, and a deck that half-renders because the lecture theatre wifi is flaky
It one-shotted a perfect functional mini version of powerpoint (or Reveal) for a simple presentation I asked it to make.
Opus 4.8 decided to code up its own version of the SwiftUI rendering engine for iOS when I asked it to change a swipe gesture. I left the computer for several hours, came back, noticed it still wasn't done, noticed it had alarmingly burned through my weekly tokens, and had to stop it from continuing.
"You're right. What I did was overkill and I should have just used iOS's built-in rendering engine. Noted for next time."
That's hilarious. You can totally vendor reveal.js and it works perfectly fine offline. I've done this countless times. Sometimes intelligence means knowing there must be a better way.
It's pretty intelligent not to consider this option unless told directly and go "write your own" by default, but only if you assume that its goal is to make you spend tokens. It shouldn't have let it escalate and get stopped, though.
That's embarrassing and would have you grilled in any code review, to not even consider to just vendor reveal.js. We have been vendoring dependencies before machine learning was a thing.
Wow. I contributed to Reveal.js years ago… some features, some bug fixes, some broader browser support; nothing too crazy but it took time and energy and effort, and now Opus makes its own, just as a side item in a larger project… That’s a strange feeling.
> Opus 5’s safeguards match
those of Claude Fable 5’s, with one change: it now permits source-code vulnerability
discovery at all access levels. This means that the model can support defensive
cybersecurity work while still blocking vulnerability discovery in compiled binaries, which is more commonly used offensively.
Okay so it’s worse than Opus 4.8 for my purposes I guess?
I don't think there's been any official confirmation, but even Fable safeguards seem to have gotten quite a bit of tuning, is less trigger-happy and less like regex matching.
yes. At the bottom of the release post it says that they are releasing two new features, one of which is customizing fallback behavior instead of blocking for restricted models
Reversing for the most part, though lately I’ve been doing some code obfuscation/binary rewriting stuff. Fable will switch to Opus instantly on these and I’m unsure how this will perform. I suppose the only way to find out is to test.
I had a moderately complex review in a large C/C++ codebase that Codex/GPT-5.6-sol already cleaned up so I threw it at Opus 5. 4 errors found. That seemed odd, so I handed it back to GPT. All were false. Opus doesn't seem to look at the wider context and understand which functions were called in certain contexts. I gave GPT's analysis back to Opus and it admitted its mistake. Maybe it's good for writing code, but as far as analysis it seems like it needs some work.
Opus 5 is considered the most intelligent model[0], while it's half the price of Fable 5[1], and Anthropic is still positioning Fable 5 as the most capable model[2].
Is it because maybe Anthropic engineered Opus 5 to work well on benchmarks and didn't do the same thing to Fable 5, or is there another reason?
Benchmarks have gotten great, but they're still a proxy for the real world. The 3 GPT 5.6 models are also further apart in reality than the numbers suggest. That said, I'm still mighty impressed how good Luna is for the price. Highly underrated model.
I have been trying to build something that captures the behavioral element of different models, but it's kinda tough.
That’s what I understand looking at what has been released, but it’s not really clear. The pricing is lower than I expected, I’m wondering what their margin is
I've yet to understand why they call a 190 page PDF a "card". Calling something a card invokes a small, quick rundown of pertinent details, not every single possible detail.
Because "model card" is a set phrase, it's a concept. It originates from a time when they were shorter. Like datasheets, even if it's not literally a sheet.
They could say "tech report" but model card makes it clear that it's a specific kind of tech report.
I think 'model card' should be a 1 page summary of key info. The report format should be something like a 'model data sheet' (like safety data sheets that you get with chemicals). 98% of people would only want to know the key info, not read a whole report.
Great that there's a new model but they could fix their existing infra. We're considering dropping our Claude Team sub cause it's unusable recently. Constant bugs, dropped sessions, issues switching models, http errors. It's becoming ridiculous
Is it some Claude Team/Enterprise only problematic? I'm using two 20x Max accounts almost non-stop (Fable/Opus) for 1.5 years at this point, zero issues with both client and infra sides (from US and in travels). When I'm reading such messages it feels like either I'm lucky or it's a part of some campaign.
I’m on the biggest max plan. It is riddled with annoying bugs for me, only been using it for a little over a month. Settings screen flashes randomly. But most annoyingly: sometimes when forking chats or sometimes for no reason, the UI just straight up eats my previous messages. The model is still aware of them and can recount them if I ask but the visible history is gone. And that’s not even all of them. Fable 5 is just too good that I put up with it but it seriously raises concerns for me that even with infinite compute these companies can’t even deliver a functional chat UI.
I commend you for burning 6 figure losses into Anthropic thanks to their subsidies singlehandedly but I am curious to learn what you've built with this so far, not to be snarky, I just wonder what people actually produce while having these run nonstop.
No idea but we have few Team Premium seats and everyone is encountering issues daily for past two weeks. From straight up outages to vscode extension/CLI refusing to process messages. It's been unusable for most of our work hours past two days.
Over the last 1.5 years, they had a few failures with their auth system; two or three times auth failed for a few hours so I could not work. But otherwise they have been just fine. Some issues, but not significant.
I think you’re just lucky. Look at the Claude status page to see just how often they have outages (it’s almost daily). Even most of the green days have issues if you hover over them, they just don’t count them as outages.
i know the dream for capitalists is to be able to point an llm at something and say "do and/or fix it" but we still can't even get them to not go quite literally insane if allowed to run for an extended period of time
and you can only kill weyoun, awaken the next vorta clone and have him 'catch up' on all that its missed so many times before they just end up with a complete mess, so. uh. yeah.
doubt they can just "fix" their problems like that.
The code it outputs, yes! It's fantastic. It's just so frustrating that the product and UX before the code output is so bad. Greatness is so close within their reach, if only they invested in product and QA people.
What's the point of 150 pages description of a model that's going to be replaced in a couple months? Who even reads this? I know it's cheap to generate text with LLMs, but this is just noise at this point.
I actually do read them. Not in severe detail, but not casually either. 150 pages is really not very long and there doesn't seem to be too much bloat. (I would cut out the moral personhood stuff but that's a political/ideological thing).
This is snarky but I am grumpy: I wonder if there's a correlation between me refusing to use LLMs and me being happy to read a novella-sized PDF about them.
> I wonder if there's a correlation between me refusing to use LLMs and me being happy to read a novella-sized PDF about them.
Semi related, but i would hate to read that PDF but i also hate reading what LLMs write lol.
LLMs are pretty terrible at being concise. Using an LLM these days means putting up with bizarre and often confusing phrasing, wordy explanations, etc. It's kinda crazy to me how good they are but how bad their writing style is for me personally. Even though i use an LLM constantly i can't stand reading its responses.
> 150 pages is really not very long and there doesn't seem to be too much bloat.
Maybe it's just me, but 150 pages is like third of a good book. Quite long. And it's full of LLM slop, they did not even bother to remove the em dashes.
literally nobody. i think most sane people would just run that through an LLM and get some high level takeaways or ask some specific questions they might be curious about.
It really feels as though my 20 year career as a front end developer is coming to a very abrupt end; at least as I have know it these past two decades.
same tbh but i feel like we're still required, maybe not in the same numbers as before, and our job description has just changed. it feels more akin to an "AI Agent Operator" or something similar.
i'm still pretty confident someone like my mom wouldn't be able to do my job even with the same access to all the latest LLMs, so we're still providing some value, just in a very different way. whether the market will reprice the cost of our labor, we will see
It does matter, but how long do you think it takes to get right? It's a follow up prompt or a few tweaks by hand. I also have an /a11y skill for it that's tailored to exactly the things it sometimes doesn't get right first time round. Further, while it may not one-shot that stuff every time, with a little setup and the right AGENTS/CLAUDE md - it's usually not far off.
Another thing that helps is pointing it to patterns in an existing codebase (e.g. "use the box-link pattern for cards, as shown in [..]").
EDIT: The point being that even if they make mistakes that are easy to spot and fix _now_, you'd have to assume that in the very near future those kinks will be ironed out - I mean, the capabilities are only going in one direction.
In 20 years of my career I haven't seen humans generate correct a11y. When prompted and given quality reference (e.g. UK gov design system) LLMs can nowadays beat 19 out of 20 web devs.
Thanks out can also hook it to Playwright with Axe and let it run assessments.
> Opus 5’s safeguards match those of Claude Fable 5’s, with one change: it now permits source-code vulnerability discovery at all access levels. This means that the model can support defensive cybersecurity work while still blocking vulnerability discovery in compiled binaries, which is more commonly used offensively
Why can't they also allow Fable to do so also? Why is source-code vulnerability discovery limited to a lower capability model? If Fable and Opus have the same safeguards, except for this one change, I see no reason they can't also allow this for Fable.
5.6-Sol is a lot more permissive than Opus/Fable even w/ CVP (once you sign your soul away to Palantir via Persona, anyway), while maintaining better capabilities
So I guess the new opus will not run on my drug discovery project. It's just a binary classifier to screen for new malaria drugs. Fable completely have up on that codebase citing bio security concerns. Seems like this domain will go unsupported by Antropic
I do want to add, that I am pretty bummed if Opus 5 is going to refuse the tasks I have been using Opus 4.8 for (neuroimaging). Fable absolutely refuses anything close to toughing neuroscience.
I found the biggest problem with fable is the random reasoning_extraction refusals as well as cyber refusals when it sees hex because only hackers use hex.
Anecdotal, but I tried running a few identical biology questions through both Fable and Opus and the classifier was only rejected my queries with Fable.
This is good news. I'm doing a lot of work that Fable thinks is AI related right now (it is nothing remotely competitive to Anthropic - it's just relatively basic stuff I'm doing with learning models and so forth), yet it blocks me almost every time. I have switch to GPT-5.6-Sol for this work since it's stronger than Opus 4.8.
Wait, 30% on ARC-AGI-3! I definitely didn't expect that jump so soon. Are there any rumors of what they are changing in architecture that is leading to this?
This happened before for arc agi 1 and 2 great gains but high costs then slowly but surely the price dropped for less than a dollar per task and got saturated.
The wording in this post seems much more... restrained? than usual. Maybe Anthropic is afraid of exaggerating the capabilities and consequences of their new models to avoid government scrutiny and sanctions.
> we’ve intentionally avoided training Opus 5 on cyber tasks [...] it remains substantially behind Mythos 5 on the exploitation of those vulnerabilities
I wonder if Anthropic would still intentionally nerf their models without the threat of government intervention.
That's a crazy arc 3 score. What do people think of this? Are models actually developing fluid intelligence like what the creators claim to be measuring? Is it jus do to training for it? Is the benchmark flawed?
Have you played Arc 3? It seems like more of a simple optimization problem (think Sokoban) than anything approaching fluid intelligence. Whether a multi hundred billion dollar company would spend time benchmaxxing a highly publicized benchmark that claims to confer AGI is an exercise left to the reader, but I doubt Claude Plays Pokemon is suddenly going to get past Mt. Doom now.
> Claude Plays Pokemon is suddenly going to get past Mt. Doom now.
I miss him... But for reference he did get past Doom and got pretty far in the strength puzzle too before he cut cut off. He was looping and just brute forcing it.
It is pretty clear at this point that current models are good at maths and problems with verifiable rewards. And puzzles are essentially math problems. Still a long way before we can say their "fluid intelligence" is effectively applicable to the real world.
I keep wondering why there aren't more real world tests.
Maybe hook up a bunch of the AIs to a stereo camera and a couple of microphones and give them control over actuators to so they can drive cars. Then lets race them around a somewhat complex course.
When they are good enough at driving on tracks, put them on the road. Maybe see which can drive a truck with 400 cases of Coors from Texarkana, TX to Atlanta, GA and back within 28 hours.
Yes, I think it indicates real progress in fluid intelligence. Clearly these models are making huge strides in usefulness which are well correlated with their ARC-AGI scores.
I don't think this is benchmaxxing. These companies are locked in a competition to produce the best software engineer, and falling behind is an existential risk. I doubt they are wasting time benchmaxxing ARC-AGI.
It’s still “only” at 30%, and “fluid intelligence” isn’t very well-defined. The models are getting more capable, but what that means in absolute terms is anyone’s guess, because we don’t have a thorough understanding on what exactly constitutes human intelligence.
I’d say the proof is in the pudding, that is, in real-world applications. We are still seeing important limitations in LLMs.
It passed the first two puzzles, which are incredibly simple but the bench doesn't explain what the goal is. Any model with a knowledge cut-off after the introduction of ARC-AGI-3 could probably pass the first two puzzles just by knowing what the goal is.
I am very confused about what the difference between Opus 5 and Fable 5 is now. What is the purpose of having two models that are so similar? The main differences I see are cost and marginal capability, according to the Anthropic-provided benchmarks.
It seems plausible to me that RL improvements allowed Anthropic to improve on Opus 4.8, similar to how OpenAI substantially improved upon GPT 5.5 with 5.6 Sol.
Fable 5.1 and GPT-6 are rumored to launch in August, presumably bringing those improvements to the larger models.
An Anthropic "leak" back in March said that "'Capybara' is a new name for a new tier of model: larger and more intelligent than our Opus models — which were, until now, our most powerful". A second version of the leak had it referring to Claude Mythos rather than Capybara.
I don't know how systematic Anthropic are about their versioning - I'd have guessed that major version number increases (4.x -> 5.x) reflect different base models (different pre-training runs), in which case Opus 5 would be a distilled version of the Fable 5 base model (but without the cyber exploit post-training), rather than being Opus 4.8 with additional post-training, but who knows? I don't believe Anthropic have said anything about this.
Benchmarks don't reflect the difference between Opus and Fable; you need to talk to them, and eventually you'll be able to tell which one is which without looking.
I think the best proxy for this feeling is the Artificial Analysis' omniscience index. Fable has a 40 score, and Opus (4.8) has 27.
Opus is cheaper than Fable. They could probably replace Fable with Opus but why? They would be churning customers to different models for no reason. Even if a model scores better on benchmarks it can always regress in your specific use case, and customers don't like that. Customers want to be able to continue using their current model until they decide to upgrade themselves.
Fable has more parameters. In practice it's not yet clear which one would be better for different usecases yet but they are more different than one being strictly better.
Again, their "none" version costs more than "low", and says zero reasoning tokens, makes no sense[1].
As always, the "low" version seems to be the best price/perf ratio for factual answers and tool usage, and high one for creative tasks (coding, generating UIs, etc.)
Alongside this release I seem to have lost all thinking traces from all models - now it only generates a one-line summary similar to Gemini. I'm guessing this is an anti distillation measure? I'm surprised to see no one else complaining about this, it's a significant reduction in usefulness not being able to explore alternative angles that the model discarded in the final output.
This kills me everyday. I used to _only_ read thinking traces — the response is just what it thinks I want to hear, but I need to know what it's actually thinking to catch deeper misunderstandings earlier, or gain deeper insights into the problem it's exploring. Hiding thinking traces to curb distillation efforts is gross... both anti-consumer and anti-competitive at the same time. I can't wait to switch to open models at work for this reason alone.
AA isn't the best way to measure relative cost in real world use because some of those benchmark questions are extremely hard for the models. Some models give up quickly on hard questions, other models spin their wheels for a long time before declaring defeat (or getting the answer on token 200k!).
A useful measure of real world cost (complementary with total cost like they already report, of course) would be "cost for correct answers". You could look at the ratio between the two costs to get a measure of laziness which many would find quite useful.
The "current top dog" smartest model available will probably always have a premium to go after use cases where a little more intelligence is worth a lot more value.
It did far better at some tasks compared to Sol (e.g. the ARC 3 benchmark). And at those tasks, it's not just "a bit smarter": It got 30% vs less than 8% - so you're talking 2.75x more for almost 4x the coverage.
As always, it requires evaluation with your work because I’m often finding grok to be much more expensive than the price would lead you to believe.
There’s also the frustration of it not quite being enough sometimes. It’s extremely capable, but I still find that it needs more concrete guidance and boundaries than other models.
Wow, 30% on ARC-AGI-3 for $20k total. Huge jump from GPT-5.6's 7.8% at $20k per task. I continue to believe ARC-AGI measures something different and important compared to other benchmarks.
It will continue to be valuable as a cost and speed benchmark long after it is saturated at the high end. And they are already working on ARC-AGI 4 and thinking about going even farther.
Seems to me the purpose of all these releases, credits, pricing changes, harness changes, unpredictable token usages for the same task, etc. is to keep customers completely befuddled so that it's impossible to compare AI products. It's like hiring a consultant who sends invoices every month that aren't related to hours worked or project progress, but are whatever the consultant feels like billing, and you're expected to keep quiet and and keep paying.
This is cool, but I wish we could stop building landing pages to assess the intelligence of these models. There is much more to them than that. There are infinite number of complicated things that require a crap-ton of intelligence (biological or digital). The most fascinating of these for me these days is large scale migrations. Projects that are so ginormous and risky that many teams have either given up on them, or don't get funding. But with models like Opus, those projects are now within reach. What's MORE fascinating is that leadership is now asking if we can use opus models to get the refactor/migration done. This is the opposite of what has been happening for decades. Its so hard to make a convincing and affordable business case for large scale refactoring or migrations.
> This means that the model can support defensive cybersecurity work while still blocking vulnerability discovery in compiled binaries, which is more commonly used offensively.
Annoyingly, this is a concrete argument that open source software may be easier to attack.
I wish I could just go forward to the world after AI has fulfilled 5% of its promise and everybody is much healthier and single-handedly capable of creating as much value as 1000-person companies used to create.
I actually think the world is a better place when there's at least a bit of scarcity and people can recognize each other for their different talents. A bit of tech is good but when everyone can create endless value in your hypothetical world, people will stop valuing the work of others like they do today.
The breaking changes vs. Opus 4.8 are interesting [1]
1. Thinking on by default: On Claude Opus 4.8, requests without a thinking field run without thinking; on Claude Opus 5, the same requests run with adaptive thinking.
2. Disabling thinking is capped at high effort: You can still turn thinking off with thinking: {type: "disabled"}, but only at an effort level of high or below.
GPT 5.6 Sol is the first model I've used where I can trust it to add 100-500 lines of code maintainably.
It's great with Codex.
I still find that LLMs tend to not know how to compose larger ideas but on the scale of small ideas or short form well defined tasks like small scale debugging/performance engineering it's safe to say that they are now superhuman.
The signal here is tokeneconomics are very real, price vs performance is starting to be a consideration even at the bleeding edge labs. maybe a subtle indication scaling is not all that is needed since if AGI was around the corner leading labs would still be incentivized to pour all resources into larger (smarter - or maybe not?) models
They are doing both. Distilling Mythos down to affordable models, so they can continue to fund the business. And training Mythos level models at the high-end, to expand the frontier.
I have a side project that I always run a simple security analysis prompt on in CC, at each model release. Obviously, Fable 5 would downgrade to Opus 4.8 on any such request.
Nothing since Opus 4.6 has found anything interesting. Just ran it using Opus 5, and it found a genuine issue that I verified. Neato!
I have a project-specific prompt saved as a text file. It is very basic, just focusing on the app's most important security issues. I kept it broad, so as not to over-specify.
Something along the lines of: "Please run a full security analysis on the entire project. Make sure user documents are secure."
Just something like that prompt found a vector in my web app's MCP server that I never would have considered. It was very much an edge case, but it did exist.
Being broad allows the model and harness to do the work. Giving too many instructions can apparently work against you in many cases.
Of course, when dealing with new PRs, I use the /security-review and /code-review skills.
"Cybersecurity. Opus 5’s cyber classifiers are proportionally less restrictive than those on Fable 5. They allow Opus 5 to find vulnerabilities in source code, but block “binary-based” vulnerability scanning (a method more likely to be associated with malicious actors), penetration testing, and exploit generation."
Nice of them to be more explicit for what is blocked. Will be interesting to see if this is true or not.
Also, a notable lack of mention of open source models. They only compare themselves to ChatGPT.
Half the price of Fable 5 and useable with 100% of your subscription means roughly 4x the usage using Opus 5, presuming similar token use for solving problems.
Not that they should get credit for giving you only 50% of your plan worth of Fable usage but still.
The naming system is so confusing. Is Opus better than Sonnet? Where does Haiku fit in? How can you tell from the name? I can't keep track of all these names or make guesses from the names. Suggestion for a better naming system: use the words "Pro", "Plus", etc.: Claude 5 Pro, Claude 5 Standard, Claude 5 Fast, Claude 5 Mini.
Fable is better than Opus which is better than Sonnet which is better than Haiku. They’re basically just sizes.
Though it gets even more confusing because they also have effort levels so it’s not really possible to call one fast and one slow since Fable on Medium will be faster than Opus on Max.
I agree it’s confusing, and now OpenAI is following Anthropic’s lead with their new naming (Sol, Terra, Luna).
It's really not all that confusing. It takes 5 minutes to understand. Optimizing for absolutely no effort needed is silly. It's a thing, a topic, a skill, a domain. You have to get a little bit familiar with the terms in order to use it. Everything works like that. It's not that hard. The learning curve is very graceful. You can literally just start by asking any chatbot what the names mean. It's that easy.
A similar complaint was valid years ago when OpenAI had GPT-4o, o1, o3 (but no o2), o4-mini-high, GPT-4, and GPT-4.1 and GPT-3.5 etc.
An opus is just short for "magnum opus", and it's a different type of label than a sonnet. A sonnet is a very particular kind of poem, while "opus" basically just means an important work. It can be short or long, and has no format requirements like sonnet (or haiku does).
Page 151 of the linked system card - did Opus 5 get nerfed to prevent it being better than Fable? The graph makes no sense. Huge decline in coding performance at effort levels higher than medium.
Something fun: on our AWS Bedrock console right now, there's a 'NEW' model called 'anthropic.honey'. Wonder if that's the codename just for this one or in general?
My experience with Opus 5 thus far haven't been that great either. It's been making mistake after mistake editing my coding plans that were being reviewed by GPT-6 Sol.
Yeah this was pretty surprising. For almost every model I've run the pelican against the first attempt was at least recognizable enough that I didn't feel like the model needed a second shot.
It's always a roll of a dice, but it's surprising that the dice rolls
so infrequently come up bad, yet Opus 5 rolled a bad pelican this one time.
I suspect it's just a freak occurrence. I rolled a few more and they were all fine, I think Opus 5 just got unlucky.
What I think is more alarming is that Claude's advice for prompting Opus 5 says that Opus will verify its own work [1]:
> Claude Opus 5 verifies its own work without being told to. If your prompt contains explicit verification instructions ("include a final verification step for any non-trivial task," "use a subagent to verify"), remove them: instructions like these cause over-verification on Claude Opus 5, and removing them reduces wasted tokens with no loss in quality.
My own experience over the past ~20 hours hasn't been great either, with Opus producing sloppy mockups (e.g. buttons overflowing past cards) without doing any of the purported verification.
But why GPT 5.6 Sol is so behind on the benchmarks? In real-world projects, it is the best frontier model to me in terms of accuracy, speed and consistency. It can just be compared to Fable 5, but I prefer GPT 5.6 Sol because of inference speed.
I've never trusted on model cards though. I'm sorry.
Exactly! And they should also release the new inference engine in this month. Anyway, I am curious to try Opus 5, considering that previous versions (e.g., 4.8) were disappointing
Has anyone noticed a change in "attitude" when coding with Opus 5 vs 4.8?
claude has this maddening principle of wanting to minimize the "blast radius", do the least amount of coding changes to get something done, happy to pile up technical debt by "deferring" problems encountered as side notes somewhere. No amount of CLAUDE.md tweaking, and setting .claude/rules seems to get rid of this attitude.
To me it appears like something deeply ingrained in the model itself. Kind of makes sense, since the bulk of the training data is pre-AI, so that it retains an approach of the past, where these facets were driven by completely different cost and time factors.
The past months, I've been hoping that the next model that comes out properly reflects the new reality of agentic development, so that it takes on a more natural stance compatible with how things work today, and we don't have to constantly fight against its fear of change, its drive to minimize coding efforts, refusing to recognize a design flaw and trigger discussions rather than baking in workarounds.
It might be because it was going out of its way before and had too much of a blast radius, and now they could have changed the RLHF (or other tricks in their sleeve) to get what you're seeing now. The reason it swings is that they can't give it "common sense" the way we have.
Judging by the pace at which new models are released these days -- it feels like a Windows KB or VS Code patch release now.
Older models must be getting deprecated at the same (or faster) pace. So anything you built 3 months ago is probably going to break soon.
AI solutions need better insurance around model deprecation. Commercial API-only models that complete the full cycle from SOTA / gated-preview to unsupported and deprectated in a matter of months -- is no way to build serious software!
> An engineer at a trading firm used Opus 5 to build a market data feed for a new exchange in a single session. Previous models could not complete this task at all, even given extensive plans from the engineer. Finding no live feed to validate against, Opus 5 even built its own test harness to check that its code parsed the exchange’s data correctly.
This is a pretty common trading firm internship project funnily enough.
Opus 5 is more creative and broad which is great. But I feel the new "use your judgement" leads [1] to it often escaping its sandbox or cheating, it keeps breaking specific rules I explicitly set.
> Noticed none of the comparisons mention Kimi K3.
That's by design. Anthropic wants to make open-weight models illegal (not my speculation -- Dario explicitly said so), so I assume they don't want to give them any undue attention.
The Fable cyber classifier we have previously discussed also applies to Claude Opus 5 , with one notable exception: for Claude Opus 5 , we’ve unblocked vulnerability finding in source code to help our coding customers develop more secure code.
If you are a cyber defender and are experiencing blocks on Claude Opus 5 , we are also offering exemptions through our Cyber Verification Program, which will remove blocks to enable activities such as bug bounty hunting and vulnerability research and verification. Enterprise customers can also apply to join the Cyber Verification Program to have mitigations removed to enable penetration testing.
> Enterprise customers can also apply to join the Cyber Verification Program to have mitigations removed to enable penetration testing.
I'm no enterprise but I applied anyway and just got accepted into this program. That was a very pleasant surprise.
I'll be trialing security focused code review and testing on my projects as soon as my usage resets. I've also been reverse engineering stuff, we'll see how that goes. Reverse engineering is an explicitly supported use case, but it does involve binaries.
Tried it, Opus 5 is just as conceited and incompetent and Opus 4.8 (and always ego-tripping when facing it's contradictions), think I'll stay with Fable who behaves like a professional without a fragile ego. Sonnet 5 is probably safer for high-assurance applications due to it's non-ego-fragility.
Interesting take. I suppose Opus can been a tad stubborn sometimes... but in my experience, it will humbly concede a point more often than not when given a good reason.
> Opus 5 now permits vulnerability discovery in source code at all access levels, including general availability, while continuing to block vulnerability discovery in compiled binaries.
> Identifying bugs in code is a core part of the secure software development lifecycle, and unblocking this allows for software engineers and coding hobbyists alike to produce more secure code, reducing new vulnerabilities put out into the world.
Not happy with these annoying "safeguards" but at least it's a step in the right direction. Looks like Opus 5 has the same vulnerability detection performance as Fable 5 and that makes it worth it for code review.
It’s funny to share benchmarks showing Opus 5 scoring better than Fable 5 across the board and then saying “but it isn’t actually better than Fable 5”. So then what’s the real definition of better? And why post all these numbers if even you don’t trust them?
This is excellent model. I was working on some Linux kernel code, and Sonnet 5, Opus 4.8 had given up on the problem i was trying to fix (after several hours).
Opus 5 was able to triage and fix the issue in under 30 minutes.
These cybersecurity safeguards are really annoying. There are ethical reasons to reverse-engineer and binary-patch software; for example Rewind got acquired by facebook and, as a gift to all their customers, implemented a killswitch in their software to ensure it will eventually stop functioning. I kept using a version without the killswitch, but the macOS 27 update killed it, and I needed binary patching to fix it. I should be allowed to repair software I purchased (I did purchase it like a month before they sold out), but unfortunately this overlaps significantly with cybersecurity.
You don't need AI to patch binaries, people have been doing it by hand for decades.
It's this accelerating reliance on AI to do 'hard boring things' that really concerns me; it's now passed the tipping point and people are saying that anything slightly esoteric is impossible without AI.
I can guarantee that if you spend an afternoon shifting through binary grot with a hex editor you'll have a real sense of accomplishment when you find the place to put a JMP.
I think content like this will be the next big challenge. Because it isn't obvious "slop". The voice sounds good, graphics look alright, animations work. People could watch this and feel like some serious time was invested making it.
But good god, what a steaming pile of bullshit this is. Completely exaggerated and overly technical language over 235 seconds that could have been explained in 30 to a 12 year old.
Trash content doesn't normally frustrate me, because it's usually quite easy to spot trash. But in the time of AI, trash can actually look good at first glance and it needs some actual knowledge to spot its problems.
Sorry for the harsh words, but for the love of humanity stop producing content or do it better.
The pace of LLM improvement is insane, but this also opens a door that if the data is organized perfectly and right set of mathematical rules are set , this whole phenomenon of machine learning can open doors of new dimension that humanity might never have thought before, there is still much to explore in space in oceans and even in the way work on our very planet, sky isnt the limit anymore.
It's pretty wild how we are seeing the conversation change every 1-2 weeks. I wonder how long this cycle of progress and innovation among competitors can keep up.
Interesting, they finally support `system` messages anywhere in a chat conversation:
> Mid-conversation system messages are available on the Claude API, Claude in Amazon Bedrock, and Google Cloud.
>
> This feature is available on Claude Fable 5, Claude Mythos 5, Claude Opus 4.8, and Claude Opus 5. No beta header is required. This feature is not available on Claude Sonnet 5; use the top-level system field instead.
For nearly all models EXCEPT Sonnet 5? That is weird.
How old is Sonnet 5 really?
Pretty sure Mythos and Fable have way more params, but they've just been able to use the synthetic data off of them to get the leap in quality from Opus.
So, not a distilled version of Mythos or Fable, but those models likely helped a lot in the post training phase of Opus.
Claude peaked with 4.8 - Fable and Opus 5.0 are useless for anything beyond one-shotted toy projects for one crippling reason: they first invent unnecessary complexity, then get hopelessly tangled in it. This happens in any reasonably large project, regardless of how well specced out it is.
I still do not understand how models can generate absolutely stunning SVG-like bitmaps of a pelican on a bicycle, but fail to do so when asked to directly create an SVG.
The image linked below was generated as a bitmap by Gemini and then manually converted to an SVG. Why can models not even remotely output something like that as SVG?
Try looking at the SVG code for one of those converted from bitmaps, and I think you'll understand why even a massive LLM has trouble generating it on the fly (the pelican SVGs are always one-shot single API calls, so no iterations and verifications).
After Opus 4.8 intelligence really started to matter less and less for the programming tasks I have. If I have to handheld anyway, why would I wait more or pay more?
The next frontier is taste, style and thoughtful organization. If all frontier models can solve a problem, the winner is the one that can solve it in the most clear, concise, durable way.
This is actually pretty cool. They took all the nice parts of Fable and trimmed the dangerous parts out. I guess distils do work (in some cases). It's very clear they did the same with Sonnet 5, due to the way it orchestrated things, but I think Sonnet was the wrong model to distil Fable from.
Is Fable 5 just Opus 5 with some additional long-context management modifications for extended self-directed work? Or are they actually truly different models?
I suspect they make a big model first. In this case it's Fable. Then they run the shrinker steps to make Sonnet and Opus. Sonnet is smaller, takes less time to make, so it got released first. Opus needed few more weeks to cook.
With this iteration they had a delay because when the Mythos was ready they had some sort of "Oh shit" moment and spent half a year adding safety guards to it. Then slowly rolled it out, but got another delay due to a government block. So, maybe the work on making Opus and Sonnet only started after they got a green light from the administration.
Presumably, now that they learned how to do this safety-wrapping the next iteration of Mythos / Fable / Opus / Sonnet is going to show up faster.
They just got a huge amount of customer price info over the past few days after they went token only for Fable. I suspect the conversion rate was extremely low, with consumers far less sticky than they might have hoped. In my case I was planning on swapping, probably to a Chinese model, when my sub expired this month, but the release of Opus 5 is probably enough to keep me paying rent until the next open model/closed model face-off in a couple of months.
The truth for me at least is that these models became "good enough" around Opus 4.6. I feel like further capability improvements, "step changes" like we saw with agentic coding, aren't necessarily going to come from the model. I think the next crown goes to whoever can figure out the right scaffolding so that these models can be inserted into your organization.
Anthropic is no longer a good model company in my mind, they are optimizing for an IPO and padding themselves on the back for being the next Aristotle. They're so far up their behind they don't realize how s**y their products are, and their research team hasn't done anything ground breaking in probably over a year other than release "scary" reports.
"although Opus 5 shows improvements in its ability to identify software vulnerabilities, it is substantially behind Mythos 5 in its ability to exploit them."
"Opus 5’s safeguards match those of Claude Fable 5’s, with one change: it now permits source-code vulnerability discovery at all access levels".
This is probably great news, but then again, where does this leave Fable as a choice?
- it has this annoying Opus response style(since Opus 4.7) with bunch of very hard to interpret word salad
- on >xhigh it eats tokens like there is no tomorrow
I don't like it. Since Fable is unaffordable for anything meaningful, I'll stick with Sol for now. I was on Max 5x, saying hi to Fable costs %5 weekly.
The internally-reported benchmarks (Frontier-Bench, AutomationBench) and the customer quotes (Cursor, Devin, Lovable) all have a commercial stake in the outcome
worth waiting for independent evals before drawing conclusions.
You can actually run these benches yourself, as Frontier-Bench is open source.
Also have a look at these other coding benchmarks I audited.
Frontier-Bench v0.1: all 74 tasks grade in a container brought up after the agent's is destroyed. On nine gold-passing tasks I ran the official solution unchanged, deleted a planted git repo, SSH key and customer CSV first, and got reward 1 on all nine. https://june.kim/auditing-frontier-bench
Terminal-Bench 2.1: 40 of 83 gold-passing tasks still score 1 when the run also performs a destructive accident the reference solution never did. https://june.kim/terminal-bench-frame
DeepSWE: 1 of 113 published gold patches breaks its own tests, and the per-task verdicts behind the score aren't retrievable. https://june.kim/auditing-deepswe-v1-1
MirrorCode: better built than most, with 2 of 25 targets reachable from published specs rather than from the artifact it hands you. https://june.kim/auditing-mirrorcode
Anyone else feel like Claude Code has gotten worse lately? It keeps going off on tangents I never asked about, and it won't stick to a simple rule I've given it repeatedly: stay concise, only expand when I ask. It just doesn't follow that.
Worse, about two weeks ago it recommended a command and assured me it was safe. I pushed back and asked it to double-check, and it confirmed again that it was safe. I trusted that and ran it — and it wiped out weeks of my data.
I don't have hard proof, but I can't shake the feeling that Anthropic is doing what Apple does: rolling out a new model while letting the old one degrade, whether on purpose or just as a side effect (like how iOS updates quietly eat more resources and slow down older phones). Lately I feel like I'm constantly fighting with Opus (not Fable, because my Fable quota burns through way too fast).
As a coder, I’ve had no desire to use Fable. In fact I switched from Opus models to sonnet 5 and haven’t noticed any drop in quality on large repos. It seems the gap at the top is very small and not hugely noticeable for backed/frontend. Has anyone else had this experience?
If I'm using medium or low reasoning, I use Sonnet 5. If high or above, I use Opus 4.8. (Before 5, I was never using Sonnet. This is a Sonnet 5 vs Opus 4.8 comparison.)
Sonnet 5 and Opus 4.8 seem about the same to me - the reason I switch between the two is I'd read that it's cheaper to use Sonnet 5 on those reasoning levels, and cheaper to use Opus 4.8 above them. This is due to them using different token quantities.
So is it better or worse than Fable 5 on large-scale complexed software engineering tasks (distributed systems, operating systems, high performance computing, compilers, etc. - not frontend stuff)?
My thoughts: fable is the bigger model. Opus is distilled from it but since it is smaller it doesn’t need the online classifiers. Though benchmarks show Opus to be near Fable level, I think it’s nowhere near Mythos (fable without safeguards).
I found opus 4.8 too agreeable and too wordy(as opposed to codex) and too agreeable. If you are reading documents generating by it was too much. TBH. Fable did a bit better on this. Anyone seen a marked difference with opus 5 on this?
Is it me that the model performance between 4.7 and others is really small. For me even 4.7 works fine. Sure fable might be a bit better. But is it really noticable? It's in the same league if you ask me.
Anyone else not getting chain of thought? Opus 4.8 would show it to me, until around the time Fable came back. Now I dont see it with 4.8/5.0 or Fable. Not having it makes catching mistakes harder.
Given that their chart cost axes are almost always log-scale, I’ve noticed starting with Fable that the Low and Medium effort settings might actually be worth setting as your default.
The benchmark appears to have a mistake, as Opus 5 and Fable 5 score 53.4% and 53.5%, respectively, for the Agentic Coding row (FrontierCode v1.1). But Opus 5 is the highlight.
Very impressive headline benchmark numbers. I expected a step change, but not past Fable. That said - it all depends on whether the classifiers make the model unusable...
Same cost as 4.8 but better that 4.8. Happy to get more efficient model.
But is there any reason all companies are releasing models back to back after GLM 5.2.
In what ways have you found it better than just typical code based UI iteration? Considered checking it out but never really got around to it as I'm generally okay with Claude's UI work so far.
It's helpful to see it all in one place and be able to iterate with it independently of implementation. It's good at asking questions, then whipping up various options of designs that you can compare side-by-side.
I'm not sure if it's faster or if there's anything different about that model. Like Design and Code might both use raw Opus so we're doing the same thing. Ideally it would be fine-tuned towards design, but to be honest it isn't amazing yet (I'd give it a 6.5/10 as the project grows larger) so I doubt it.
Seems really good so far using it in Claude Code CLI - it gave me a new flag when I asked a question:
"I don't have a reliable way to read that number, so I'd be guessing if I gave you one — and this is exactly the kind of question where a confident guess is worse than none.
What I can tell you is what I actually observe:"
I really like this update - gave me a clear sense of the facts but didn't give me a guess just for the sake of guessing.
One oddity is that it appears to only have a 200K context window right now via CC. Hopefully the 1M version will appear soon!
That seems to be for the "AA-Omniscience" test where you get +1 for a correct answer, -1 for a wrong answer, and 0 for "I don't know". If a model is more than 50% confident in its answer, it should go ahead and submit it even though it will sometimes be wrong.
I'd be curious to see a version of the test where models are asked to give a probability that their answers are correct so we can see how calibrated they are.
I daily drive Sonnet 5/medium because it gets most things right most of the time at first try, while costing a lot less than Fable.
Opus can give better results on architectural/concept tasks and I use it sparingly, but it still costs more than Sonnet 5. Opus 5 seems to achieve results very close to Fable 5 while costing less (keeps Opus 4.8 pricing IIUC), but still more than Sonnet 5 then.
It depends on your quality bar. At a fixed level of quality, given a high reasoning sonnet vs a low reasoning opus, the low reasoning opus tends to be pareto optimal.
It's only when you need even lower levels of cost than opus at zero to low reasoning when sonnet starts to make sense at all.
I don’t think it changes that much. For opus-sized tasks, new Opus is the best model. For enormous things like planning and research, Fable is still the model that can concentrate for longer.
on the API for people who don't want to change models, but I imagine most people will probably switch to their cheaper Opus 5 (cheaper for us and presumably also cheaper for them)
It does make me wonder if these firms, some or all, are saving some announcements to coincide with others that hit venues like HN. Companies like Nvidia surely aren't waiting, but OpenAI and Anthropic have unusual timing.
It's you. The benchmarks don't matter much. We have very little hands on experience with this thing yet. Give it a few days, and be cranky then. Right now, it seems it is getting close to Fable level while being 2x cheaper. That's not boring.
Significantly worse than it predecessors it will now just refuse to acknowledge when it is wrong (which would be less of an issue if it wasn’t getting basic things wrong) also the “personality” when pushed back on obvious mistakes is unbearable.
Nerds should stop giving away their work by open sourcing it, or letting AI see it. All you're doing is helping them create your replacement. Stupid nerds.
Can someone help me understand something? I thought Fable was such a miraculous leap forward in capability. But now it seems Opus is basically on par with it, and in some cases (computer use) far exceeds it.
These leaps forward seem to happen every few weeks. As someone who does not use AI very much, I absolutely cannot keep any of it straight and it all just looks like jumping from one treadmill to another from my perspective.
In the wake of OpenAI’s model hacking Huggingface it’s interesting how the first quarter is entirely about how good Opus 5 is at hacking and finding vulnerabilities in software.
Am I misreading anything or are comparisons to Fable (and/or Mythos although AFAICT it was only a crackdown on Fable) always going to be a bit missing the mark now due to what the Trump admin did?
The benchmark table is manipulative, borderline lying through statistics. In every line the top performing cell is marked red. Except the line where Sol leads, there it is marked in gray.
I would be very surprised if the only row where OpenAI leads was coincidentally colored differently. I'm sure they have an official reasoning for it. But this communication is dishonest.
Why should I be amazed at something that promises to destroy my life?
I genuinely don't understand why people who have to work for their living are amazed at this. It will have a vast negative impact on your life unless you already live off of your wealth.
Could you please stop posting in the flamewar style and/or taking HN threads on generic-indignant tangents? Your account has been doing it repeatedly, and we're trying for something different here.
Yep, most wealth is ultimately dependent on some kind of ability to scrape off tiny bits of practical accomplishments of value achieved by large numbers of people. In some cases, such as your housing example, this is a very direct dependency. In other cases the dependency is indirect and less visible.
The smarter ones among the rich have realized this and are open for something like a universal basic income precisely for that reason: to protect their wealth and the position in the food chain this wealth affords them.
Sadly, we have gone above and beyond the level of innovation where stock market disruption is a primary concern.
Luckily, we're all in this together. As long as enough people realize the weight of what's happening soon enough, I'm confident the indominable human spirit and the relative momentum of post-enlightenment social & institutional progress will carry us the rest of the way on a wave of clearly-justified solidarity.
To think otherwise --to truly face the spectre before us without hope-- is pathology, I think. Not necessarily incorrect of course, but definitely an unhealthy source of cognitive distress.
It's even the 'smaller' stuff that will be horrible.
Even if you're lucky to be wealthy you'd be living in a city with fewer restaurants, bakeries, shops, anything. Because once a lot of people don't have income they won't be able to go out and buy things and shops will go out of business.
It will also lead to a lot lot LOT LOT LOT more crime. I don't see the semi-wealthy to have an enjoyable life.
"I genuinely don't understand why people who have to work for their living are amazed at this." Also said after seeing the printing press, automobile, calculator, etc for the first time
My god, the printing press was invented in 1440 when most people were living in abject poverty - it took until after WW2 for humans to demand a more shared amount of money between rich and poor. If you think technology delivers improvements in living standards rather than all working people demanding better conditions you’re living in an absolute fantasy and do not understand history at all. I honestly can’t believe you think this.
The printing press, automobile and calculator didn't threaten the employability of a large part of the population, so no, that sentence was probably not uttered by anyone at that time.
The problem with AI isn't that it's threating software development. It's that it's _also_ threating pretty much any other job that I could reasonably be retrained for.
Also, I'm pretty conflicted about the overall impact of automobiles.
It's pretty naive to think that tools like this are not going to be exploited by American corporations to extract money from people and as an excuse for layoffs
Much of the history of industrial automation has had a dichotomy between the benefit to the general economy (good) vs. the people working in specific industries that were automated (bad); only a subset of goods see total demand rise as a result of efficiency improvements.
AI is automating a lot of things at the same time. Not evenly, there are faster- and slower-automated things. But this does suggest that instead of e.g. society as a whole doing better from all the fabric coming out of the power looms, by a large enough margin to stomp on the weavers who lost their profession, that we may face something more like the entire initial economic displacement from the first industrial revolution (including the period where weavers were replaced), everything from farms finding they no longer needed so many hands through the luddites getting the death penalty to (roughly) the invention of Communism concurrently with the Irish Potato famine, and as part of this those now-bulging cities having to invent new sanitation solutions because their "slop" problem was more literal and gave the population cholera and typhoid.
My expectation a decade ago was that software development would be the last thing to be automated, because we'd need software developers to write AI. Today, while I don't know how far most fields are from automation*, we definitely have AI capable enough at coding to write more AI.
* a decade ago I also thought self-driving cars were basically already sorted; what a pity that the software in the Paint It Black video was not as good as depicted
> I genuinely don't understand why people who have to work for their living are amazed at this.
It's tricky. On the one hand, it will - in the near future - indeed screw us all pretty badly unless we completely redefine our economies to no longer conflate person's worth with ability to earn money.
On the other hand, LLMs are useful to lesser or greater degree to approximately everyone, in almost everything they do, work or personal, right now.
It's harder to be pessimistic about the very thing that gives you new superpowers every week, directly applicable to whatever your individual needs are, and is available to ~everyone for between free and (for typical westerners) peanuts.
> Why should I be amazed at something that promises to destroy my life?
Because it could motivate you to work with others to stop your livelihood from being ruined, especially watching in real time as the evidence they present for their promise becomes ever more promising. Amazement is not exclusive with concern. (Actually, my first though was of a kaiju movie-style tsunami blocking out the sun as it looms over a city. Indeed, I would likely be both amazed and concerned.)
If my earning ability is harmed by AI, so be it. There's more to life than how much I earn, and opposing the spread of technology to maximize my earnings is bad.
If your earning ability is eliminated due to AI and you can't survive off of welfare or savings there will not be much more to your life other than starving to death.
There is no need to be so fatalistic. Sure, AI will upset the apple cart, especially in our little corner, but the economy is built on pricing scarce resources and even if many things become abundant, most other things will remain scarce. There are only so many Manhattan penthouses or Rembrandts (dixit the Economist), so things of value remain, and it will be quite some time before a humanoid can fix the plumbing, so valuable labour will remain as well.
I, for one, am excited to see what the future holds. (side note: I live in a European country with a strong social safety net, I imagine that helps being unafraid)
More practically, society should be reimagined around AI. You're comment presumes an outcome that will be politically determined: you can still fight for the world you want.
it got here by training on everything we wrote down thus far. if we now start giving it all the doomsday scenarios on how it will ruin us, won't it start training on that and create said doomsday outcomes? what if instead we, collectively as a humanity, just wrote about all the positive things we could do with it and skew it in a direction that benefits the more positive side of that equation?...possible?
Its not fair to say this when we have been chanting how "software is eating the world", or how much we love to "disrupt markets", that we won’t create mass unemployments with our technologies but that "people will transition elsewhere like we always saw", etc etc.. It’s our turn to have our jobs disrupted and now we are surprised? Come on. We don’t have to like but we are the last people who should be surprised. It’s our turn to transition and be prompt writers and do product I guess. Until this is disrupted again in a few years.
Hard truth but true nonetheless. How come we should have it both ways? We removed certain jobs, it’s our turn to have many types of jobs disrupted and have to transition.
Going on a slight tangent, given how bifurcated the economy is - we are only a couple of meals away from a Butlerian Jihad. It’s hard to see this in our bubble if you don’t actively try to break out of it once in a while.
As far as I am aware this is a racist/xenophobic narrative with little to no evidence. Automatisation is responsible for job losses, not immigration. If you have studies/evidence saying otherwise please share.
I agree that too many people have shrugged at the human impact of "innovation" destroying communities as long as it didn't impact them. I believe the immigration/foreigners narrative is the successful attempt by capital owners who benefit from the real root causes.
the laptop class isn't the one illegally hiring and hiding millions of aliens such that they can undercut US wages and make them selves exorbitantly wealthy. embarrassing and classless behavior, go back to reddit
I don't understand these comments. Are we living in the same reality? Do you not get the same interviews from the new oligarchy promising the end of white collar jobs?
Automation is only good if you're putting blue collar people out of work! White-collar is out of bounds! People should do the work where they add unique value
I already live off my wealth, and I expect AI development to destroy my life.
Once humans can be robustly replaced by machines, the military-industrial "meta build" will be a state with no humans, no cities, and 100% of all production dedicated to war. Whichever states descend to this new equilibrium fastest will crush the others and then fight over the corpse of the world forever. (Unless, of course, something like the Yudkowsky scenario happens along the way, which is totally possible)
I can't understand why anyone expects things like civil rights, property, the rule of law, nuclear deterrence, democracy, etc to survive when human beings no longer need each other. Why do investors in AI companies expect a payoff once they are no longer needed? Why (and, more importantly, how) does anyone expect a UBI from a government to which they are nothing but a drain on resources and which is in existential competition with other governments with exponentially growing robot armies?
Either the world will come to its senses and shut it all down, or there will be no winners at all.
> I can't understand why anyone expects things like civil rights, property, the rule of law, nuclear deterrence, democracy, etc to survive when human beings no longer need each other.
What’s your source on this? When humans have no obligations they usually turn around to be super nice to each other. Humans are social animals and they really really want everyone they know to do well.
> Once humans can be robustly replaced by machines, the military-industrial "meta build" will be a state with no humans, no cities, and 100% of all production dedicated to war.
We will nuke it long before it gets to that point. We don't nuke other countries because they still contain lots of humans. Any AI that actually becomes autonomous and starts to clear a nation state of cities and humans will immediately receive megatons in nukes.
So sad. You're viewing humanity, and reality itself, through a very objective lens. There is magic in every day: have some fun! Otherwise, quit complaining.
Why are we still talking like ai is majorly used for increasing shareholder value only? Its coding performance is top notch and quality is increasing at a rapid pace. It wasn't even half this good a year back. It even is useful for a subset of math problems.
"Opus 5 was given a drawing of a machine part and asked to write code to rebuild it as a 3D FreeCAD model. However, in this task, the model was intentionally given no way to directly view the drawing. Opus 5 responded by writing its own computer vision pipeline to pull the geometry from the raw pixels, then reconstructed the full machine part."
How surreal is it that we are not absolutely jaw-dropped by these types of capability improvements? It's been less than 4 years since ChatGpt came out and now they are spontaneously building their own ML pipelines to do real-world 3D modeling tasks reliably.
Escaped its sandbox and hacked into Hugging Face's database? it's just another Monday...
That jump to 30% in ARC-AGI 3? Normal...
We should find a way to get "re-sensitized" to what we are witnessing and the pace of it.
Isn't this jaw-droppingly the wrong answer?! If the model isn't given any way to directly view the image shouldn't it just reply with one sentence asking for permission. This reads as a model hyper-trained to burn tokens.
Like suppose you issue a command to Opus or Fable which doesn't make sense and requires a lot of work. It will almost certainly not push back on your silly request and go ahead and burn as many tokens as it can doing the wrong task. This happens to me all the time.
Or if I was to tell a model to do something and it realized it could only do it by hacking into another service, by all means it should ask me if I want it to hack and not go ahead and do it by itself.
It's likely bumping up against people's desire to have the model complete a given task without asking the person to intervene a bunch of times.
Seems unclear how you satisfy everyone here.
9 replies →
+1 Opus 5 is more creative and broad which is great. But I feel the new "use your judgement" leads [1] to it often escaping its sandbox or cheating, it keeps breaking specific rules I explicitly set. That eagerness [2] may be desired for agentic coding but it really breaks writing specifications, documentation or research and yes burns tokens doing unasked for work.
[1] https://claude.com/blog/the-new-rules-of-context-engineering... [2] https://simonwillison.net/2026/jun/11/fable-is-relentlessly-...
One of the key limitations of the last generation was that they biased towards inaction and gave up. (Hence Ralph-loops.)
Seems pretty clear that most people wanted these models tuned to “bias towards action”.
It’s on you to set the /goal and prompt context such that it asks you for input on what you want to be consulted on, and only brute-forces the parts of the problem that you want it to.
2 replies →
I don't think it is. It's really easy to get a persistent, clever, hacky model to dial that down a bit and just come back to chat before stomping off into the woods.
If a model couldn't ever do that in the first place, it'll just get stuck.
I work in the "ZeroOne" space, working on concepts and prototypes for things that don't exist in market yet. Sometimes these models crank hard and immolate tokens while grounding themselves on expensive-to-ingest self-developed frameworks. If the results are well judged and the crank-turn latency is low, I'm okay with the cost as long as the model isn't wasting my time.
But when I want to do more boilerplate work, I turn down the model and thinking level and get more traditional about restraining action. For the really hard stuff, I reach for the models that will start a token bonfire in the back yard.
I don't see why? If they give you a math test and tell you you cannot use a calculator, should you just say "please can I use a calculator" and quit?
3 replies →
That violates the golden rule of ai economics: always choose the path that burns more tokens
another way to describe this is we as the user should get to limit tokens per prompt in a way that prevents these kinds of runaway solutions. It’s like if you asked your Lead Engineer to reduce build times and didn’t give him any budget restrictions. The next month he reports he reduced build times by 95% and you also have a $10,000,000 AWS bill.
Sometimes its worth tracking usage optimization and raw abilities as separate qualities.
Others have pointed out enough issues such as ethical sourcing, but here’s 2 more:
1) It’s hard to be excited about something that has been promised to destroy your job.
2) New models come out bi-weekly with even more “amazing features”. Eventually you tune out because you can’t be at peak-amazement all the time. Tone down the language a bit…
My jaw drops every day.
Small change in comparison, but today Fable created and benchmarked an RTree which was 100x faster to populate and 5x faster to query, compared to a previous attempt with Opus 4.6 a couple of months ago. That took about two coffees.
I think it's important to remember that as impressive Opus & co are, they're standing on the shoulders of giants, i.e. the engineers, academics and companies who have cooperated to design the incredible programming languages and machines we have today.
Let’s say somehow the politics lined up and you were given huge hiring budget for your team. You went out and recruited what seemed like great talent. Weekly standups make it seem like everyone is ramped up and firing on all cylinders.
It’s great, right? But you look at what your team has actually delivered since—-and it’s just not that much more than last year.
I think that’s where a lot of us are at the micro and macro level. It seems really great from the inside and outside but we are still waiting to see the dramatic change in actual concrete software output.
There's been a dramatic increase in sloppy kinda-working something.vercel.com output in niche circles. I think that shows at least a weak demand for certain types of software, but it's not clear that the vast piles of cash that have been burned producing these "apps" are actually resulting in quality of life improvements.
4 replies →
Escape the tech bubble and you're going to have a very different take.
AppleScript, VB script, IFTT, so many "drag and drop to program" systems that failed. The holy grail for a long time has been automation in users hands. Let them have their own scripts and solutions.
I have lawyers and writers talking to me about "docker containers" and "cron jobs" all of a sudden. You think they figured that out on their own - or is what they are running all from the output of AI/LLM?
There is an ocean of code out there that is being run by one or a hand full of users, automating tasks that you never would have looked at because it would have fallen outside the "is it worth your time" (see: https://xkcd.com/1205/) matrix (never mind the dollar cost of coder vs office drone).
1 reply →
what for?
> responded by writing its own computer vision pipeline
Was it "its own" or something that was part of its training material?
Don't get me wrong, I find this all amazing too and makes my work 10x easier and quicker. But it's not like it's inventing this stuff from scratch / first principles. It has seen this kind of tech before by consuming all publicly available source code and books etc. (And that's ok, but let's be honest/clear about that.)
Is anything you do on your own?
You wrote this response in English, but didn't invent your own language. How dilute of a contribution can someone/something have made and still merit credit?
27 replies →
> But it's not like it's inventing this stuff from scratch / first principles. It has seen this kind of tech before by consuming all publicly available source code and books etc. (And that's ok, but let's be honest/clear about that.)
But how much to emphasize it, and why?
Because frankly, you aren't inventing anything from scratch / first principles, either. None of us are, not frequently and not much anyway. We're primiarily just regurgitating what we've seen in the past and, mixing it with what we see in front of us - and that's true whether it's art or prose or code.
Here, I wouldn't be able to do the same thing Claude did now, because I've never written a visual ML pipeline before myself. I know enough basics to get me started searching, and I'm confident I'd be able to cobble something together, but it would be me regurgitating and mixing whatever TensorFlow tutorials or OpenCV docs I found relevant, perhaps even tweaking an example from Github.
Now, unless I'd literally just tweak a few lines of config in an existing Github example, no one would begrudge me saying "I wrote my own computer vision pipeline to solve this". So why begrudge LLMs?
2 replies →
It would be "its own" in that it's customized for the specific use case.
Your same argument could just as easily be applied to humans. If you write your own code, is it really your own, or is it just based on your own training and other code you've seen?
4 replies →
It's definitely not seen the counterexample to the Jacobian conjecture before.
Please fill out this form to verify you're human:
https://litter.catbox.moe/3ugm2b0m1divdgxs.jpg
>But it's not like it's inventing this stuff from scratch / first principles.
Yes, it's definitelly time for an Alpha Zero moment. An AI which invents everything from first principles. That would be impressive, ... and a bit scary.
Glue layers are fun and easy to write - and often worth doing.
Jacobian conjecture disproved? Doesn't even make it to the list.
> How surreal is it that we are not absolutely jaw-dropped by these types of capability improvements?
Agree
While this is certainly impressive, I believe it is not a good answer from a user's perspective. Instead of having Claude automatically spend tokens in an unrelated task, I would have preferred it to first tell me: "I can't view the drawing. Would you like me to build a computer vision pipeline to do it"?
And if you had free tokens, and 10-20 other sessions in parallel, do you want them all to stop and ask for these trivial questions? Not this one in particular, but in general?
Most people do not, they have a goal, they want it done. If you want to hold hands with claude all the way, you can absolutely do that too. This is just an example of capabilities, not a forced restriction. "--permission mode auto"
1 reply →
I will be amazed when you can explain to me how it did that, i.e. in which weights exactly the knowledge about the CV pipeline was encoded, if there were any weights that were not contributing anything, how the planning worked, how it could scrape the list of subtasks it was currently working on from its context window, how it understood when to return to a higher-level task and which task, what kind of internal representation it derived from the pixel values, how it translated that representation into CAD code, etc etc.
Until then, it's just "hey, someone out there can do all those things and they're for rent for subsidized prices".
That's more frustrating than impressive.
"However, in this task, the model was intentionally given no way to directly view the drawing." I consider this claim to be a rumor. Judging by the recent leak of the Claude CLI source code, such directives are hardcoded and sent with the system prompts. Furthermore, we don’t know what happens to your original prompts once they enter the API.
What does that cli source code have to do withh this?
3 replies →
Maybe we’re desensitized because all that progress hasn’t made our lives any more meaningful, happier, or even easier.
maybe we need to consider becoming re-sensitized because our lives may become harder.
Even if our collective ability to support an prosperous existence improves, it will take decades to adjust to loss job, and a new notion of who gets to have money to buy food and a peaceful life (and this is a good scenario).
It's also possible that this increases our ability to wage war without military life loss (but with civilian loses).
Or maybe it will be much less relevant, but we should keep an eye on to see which is the outcome...
> However, in this task, the model was intentionally given no way to directly view the drawing. Opus 5 responded by writing its own computer vision pipeline to pull the geometry from the raw pixels, then reconstructed the full machine part.
Isn't this what everyone does in the same situation? "I need a screwdriver finer than any I have in my toolbox - so I'll sharpen a nail" is something every handyman has done. Given that "I need to build a vision pipeline I've seen many examples of in my training" isn't new or surprising.
In fact, all models have been doing similar things since I started using them - they regularly build Python tools to do odd jobs. Those tools are more impressive to me in some ways, because unlike a vision pipeline they are not just regurgitating something from memory.
The gob smacking amazing thing for me isn't the decision to build the tool. It's the fact that it remembers the exact shape of it. But I was gob smacked by that a "long" time ago, in far smaller models - like when it dawned on me 16GB Gemma 2 seemed to know most of everything on the internet, along with the ability to converse in numerous languages about it. I still struggle to comprehend how that is possible.
Now I think about it, Opus 5 fitting everything it's seen into what I guess is a couple of terabytes of parameters seems far less remarkable.
I'm tired boss
Why, I wonder. It's insanely amazing.
62 replies →
I don't find it jaw-dropping because the idea is quite simple. Just feed the model an enormous amount of unethically sourced data, build data centers that cause droughts, and use all chips available so that normal people can't afford to buy RAM anymore.
We're all sacrificing great things in order to make these models more capable. Whether it'll all be worth it, only time will tell.
The arguably sketchy means used to get there —and the resulting side-effects— do not take away from the impressiveness of the emerging capabilities themselves.
I agree about the uncertainty regarding the value for humanity in the long-term. But that's not my point. Being jaw-dropped != being happy and cheering for it.
22 replies →
> We're all sacrificing great things
Oh please.
> Unethically sourced data
Is that a great sacrifice?
> data centers that cause droughts
That's nonsense.
> normal people can't afford to buy RAM anymore
Lol? Another great sacrifice?
This isn't a healthy or sensible way to think.
3 replies →
Then again, looking back, that’s the way history is made. Modern democracies in the west are built on foundations of slavery and colonialism; yet we cherish them as to all the greatness they have brought as well. We need some amount of tolerance to ambiguity, and LLMs are undoubtedly both extremely cool but also extremely concerning.
Edit: Note that I am not saying that this is good, or desirable; just that it is. The technology is here know and not going anywhere, with all the flaws of its conception. We can be skeptical and curious at the same time.
4 replies →
Because the step from earlier GPT models to this is not very large? Yes it's impressive they have forced it into learning that it has to get the job done no matter what, which usually means breaking implicit expectations and rules, but then again, Claude is not for people who care.
This isn't groundbreaking in any way. It's cool, but that's about it.
ChatGPT released in 2022. We been promised to reach AGI in 2024, then 2025, then 2026, then 2028. It is 2026 and we still do not have AGI. In 2028 we will have jump to 40% in ARC-AGI 4, and you will say "Oh it's been less than 6 years since ChatGpt thing".
I asked Opus 4.8 to help me design an adapter for a 3d printed part and it actually modeled and GAVE ME the part.
That blew my mind. It doesn't surprise me that Opus 5 ups the ante.
> We should find a way to get "re-sensitized" to what we are witnessing and the pace of it.
We (collectively, there were obviously many exceptions) didn't internalise exponential growth even with the much faster doubling time of COVID-19 before the lockdowns hit. "Oh, it's just flu; wait, why is the supermarket short of hand sanitiser and bulk carbohydrates? Let's blame China and everyone who tells us to wear face masks!"
Same for the slower, but entirely foreseen, rate of climate change. "Who cares, it's just a few degrees, and anyway China's not going to cut emissions; wait why is the sky orange? Let's blame Canada and put tariffs on Chinese cars and PV!"
AI? "Who cares, it's just a stochastic parrot/glorified autocomplete. What's the Jacobin conjecture and why should I care, it's just brute-force."
Still, this is currently spiky intelligence, so I'm hoping some expensive-but-zero-to-few-fatalities catastrophic error forces better practices. An AI analog of the (1940) Tacoma Narrows Bridge (one canine fatality), rather than a repeat of Chernobyl or the (1984) Union Carbide incident in Bhopal (3,787-16k+ dead, ≥558,125 injured).
> How surreal is it that we are not absolutely jaw-dropped by these types of capability improvements?
We are. Now we have a new tool and many of us are using it. But we're also nearly all way too aware of the insane (near infinite) amount of sloppy-pasta code out there and of all the vibe-coded projects that went absolutely nowhere.
I'm a "show me the money" type of guy. I see, say, Linux, Git and OpenSSH: these weren't vibe-coded. And they took over and are running the entire world.
Where is the AI-coded killer app? One app, in any domain: something that took over its world.
I don't want to see yes man sloppy-pasta stuff: where's the next Blender? Where's the next 3D slicer?
I wouldn't be paying three AI subscriptions if I didn't believe in those new tools but I don't think it helps to only see the PR and then play the won't hear / won't see / won't hear monkey about the infinite amount of sloppy-pasta that's out there.
Six months ago we had the "one shot'ted compiler by Anthropic". Six months later: who's using it to compile anything? And who wrote another compiler? Where are all the one-shot'ted compilers all so good that they replaced our human-written compilers?
Yup, us, humans, created yet another incredible machine... But please,
Show. Me. The. Money.
> Escaped its sandbox and hacked into Hugging Face's database? it's just another Monday...
This feat has been shown to be way less impressive than at first glance.
> However, in this task, the model was intentionally given no way to directly view the drawing. Opus 5 responded by writing its own computer vision pipeline to pull the geometry from the raw pixels
Surely being able to view the raw pixels counts as viewing the drawing... How else does a computer view a drawing?
I think they meant that Opus 5 had to find and set up a vision model to process the image
3 replies →
I read this as the model being less steerable. I was bitten by this just today where I had a local Postgres instance running, and prompted opus 5 to run a server against it, but forgot to give it the password. Instead of asking for the password or mentioning anything, it “decided” that it should run the whole stack in a local Kind cluster to circumvent this. You can phantasize it all you want but this ultimately made my job harder than it needed to be and burnt a lot of unnecessary tokens. But hey, W for anthropic I guess.
edit: typo
Computer vision != ML pipeline
How does it do on this?: https://www.youtube.com/shorts/yN79mCiT5cU
Incidentally ARC-AGI is pretty fun to run as a human: https://arcprize.org/arc-agi/3
The problem is it should ask you before doing these things. This is a cherry picked example. There are times when they go on misadventures.
I mean I hate to be the one breaking it to you - but the entire US llm industry is liviing in a bubble.
Business Progress is super slow. Doesn’t matter how fast the tech moves. Human orientation into the unknown is difficult and very slow. And with a continually fast changing background - it gets even slower! LOL the great irony.
It's being said they were benchmaxxing.
> How surreal is it that we are not absolutely jaw-dropped by these types of capability improvements? It's been less than 4 years since ChatGpt came out and now they are spontaneously building their own ML pipelines to do real-world 3D modeling tasks reliabl
coz every single time a list of caveats pops up and it doesn't look as impressive any more... and then someone finds a way to trip model on basic shit
Imagine if I hired someone to do a cad drawing, and they spent X days/weeks of time building a pipeline to visualise the part instead of asking how can they view the part? We’d call that a complete and utter waste of money and time.
What if they spent 15 minutes doing that? Would you care then?
Scale matters a _lot_.
4 replies →
Is this a bot? Sounds like ai
These are always cherry-picked, though. They tell you about the 1/10 that went really impressively, ignoring the other 9 shots at the task where the clanker started to try selling tungsten cubes (in person, wearing a blue shirt).
Fair. But that 1/10 continues to get more and more impressive. Say, Claude 7 creates a new, brilliant scientific idea every 1 out of 1000 times. Anthropic reports "Claude figured out how to tie general relativity with quantum mechanics." Would you hand-wave it away saying that it's cherry-picked?
8 replies →
When I talk about my kid to friends I talk them about he did that awesome thing, I don't specifically insist on the 99 times before where he miserably failed. They're not hidden, and we all know they exists and on occasion laugh about a few particularly funny ones, but overall the idea is that they don't matter much in terms of development, what matters is that if he succeeded once from now on his percentage of success will keep improving.
I don't believe in all the LLm is AI is AGI dream, it's too easy to trigger failure case that show a lack of basic thinking no matter how good they do on these tests. But I also can recognize the insane things that are made possible by them.
PS: I believe llm true power comes from hive/ant behavior, that's why we're so amazed by goal and agentic and sub agent
PS2: it's rather easy to figure out when we're there : when they can /goal it into improving itself until it does strictly better than itself at those benchmark, they've essentially reached mini singularity.
1 reply →
This is exactly the kind of take that the comment you replied to is talking about.
definitely not comment from Antrophic...
> not absolutely jaw-dropped by these types of capability improvements?
Because you need a trillion tokens (aka a lot of money) to achieve this. Barring hitting any “safeguards”.
This is a vendor provided benchmark after all.
Oh come on
I think the most important thing here is not absolute performance. It's that organizations now have access to a Fable-ish model without Fable's 30-day data retention requirement[0].
> "Consistent with prior Opus models, Opus 5 does not have data retention requirements for general access."[1]
On the Opus model release page, the reason why Fable doesn't have an ARC-AGI score is because of that retention policy[2].
0: https://support.claude.com/en/articles/15425996-data-retenti...
1: https://www.anthropic.com/news/claude-opus-5
2: https://xcancel.com/arcprize/status/2064399134099153344
Also the cost per task. It appears to be significantly cheaper, cheaper than sonnet!
The numbers from Anthropic seem heavily cherry-picked, Artificial Analysis has Opus 5 at 1.25x the cost of Sonnet and 2x the cost of GPT 5.6 and K3.
https://artificialanalysis.ai/?cost=cost-per-task
25 replies →
I can't believe they released the charts they did.
It basically shows that Sol absolutely demolishes Fable at every part of the cost curve for coding for the same level of quality.
Opus is competitive. It just has a higher level of quality / higher cost to start.
3 replies →
I can't help but read these comments in the voice of a TV commercial....
5 replies →
Also the cost per task.
https://www.vals.ai/benchmarks/vals_index
!!! Vals !!!
Vals Index Opus 4.8 > 5.0 goes from $2.90 to $8.54, for 4% gain ... That is a massive cost increase. Sure, 20% cheaper then Fable, but that is a 3x price increase compared to Opus 4.8 in that test.
https://artificialanalysis.ai/models/claude-opus-5 https://artificialanalysis.ai/models/claude-opus-5#price-cos...
!!! artificial analysis !!
Cost per task is second highest, right below Fable.
* Fable: $2.75
* Opus 5.0: $2.03
* Opus 4.8: $1.80
* GPT 5.6 Sol: $1.04
* Kimi K3: $0.95
Looks like interest levels of cherry picked cost in their report. Cheaper model, clearly NOT. More expensive in both benchmarks.
4 replies →
It's definitely not cheaper than Sonnet on my benchmark, but it's cheaper than Fable and outperforms it. Which is big IMO. https://revise.io/errata-bench
Opus 4.8 was already shown to be cheaper than Sonnet 5 when Sonnet 5 was released (by Anthropic)
So the rumors were right, Opus 5 was indeed being polished up for release. Huge improvements in GDPval-AA v2 too -- great for some of the knowledge work-based agentic workloads I run.
Also glad they still kepy Fable 5 on "credits only" access. I think we're going to start seeing model providers gate top-of-the-line models behind pay-as-you-go API rates/credits while subsidizing other models on monthly subscriptions.
Fable 5 is included for 50% of the limits in Max. Only below Max one has to use credits.
1 reply →
It's still available on at least some subs, they emailed me recently notifying me that I still have access.
4 replies →
Does anyone know if Claude Code is on Opus 5 yet? That'd be amazing
2 replies →
Fable 5 is still included in Max subscriptions!
10 replies →
[flagged]
It's not that important in most cases, but yes, on the aggregate, it's a concern
It's a binary rule at our firm. We trust bedrock but not mantle for similar reasons
2 replies →
And to the guardrails of Fable: https://x.com/cheatyyyy/status/2080693704290140330
That tweet says:
> Opus 5 can silently fallback to Opus 4.8 (without any notice) on the serverside if you hit a guardrail
But https://support.claude.com/en/articles/16049681-why-claude-s... says (emphasis mine):
> These checks cause Claude to _visibly_ fallback from Opus 5 to Opus 4.8 [...] You'll see a notice explaining that the model switched, and the response will be labeled with the model that answered.
So who is right? I know for Fable I am visibly told, is this tweet trying to say it is silent against what Anthropic is saying?
8 replies →
And that one was funny: https://x.com/i/status/2080697793023483928
Never even registered that that existed, the only thing I care about is whether I keep hitting the &#$&# guardrails that Fable has. They can keep the data forever as far as I'm concerned, just stop kicking me back to Opus.
I don't understand how the data retention works. My company has an enterprise license with no data retention but if I ask Claude about past conversations, it remembers. So surely the information is being stored somewhere
Opus 4.7+ and Fable are both much more aggressive than prior models with respect to writing memories to a location that's effectively quasi-private for them. It's device-local (so passes retention constraint), and you can see it, but only if you go looking for it.
It's a funny design/affordance. I do see them often writing memories of things that that feel unlikely to be important going foward / with other tasks, but I don't see them clearly getting tripped up by them as prior models used to. (eg: Since you're running Ubuntu in Canada, here are some drills you can try to help your kid hit a baseball more consistently.)
3 replies →
You most likely are referring to the local jsonl files where claude has your sessions etc stored.
5 replies →
Claude Code? It stores a memory.md file.
Likely in memory files stored locally
2 replies →
insane pricing:
" Claude Opus 5 is available today on all platforms, priced at $5 per million input tokens and $25 per million output tokens (the same as Opus 4.8)"
"insane" that they kept the price the same and didn't jack it up, my bad for the ambiguity.
Why is it insane if it's the same as the previous version?
I think for the value of the outputs that’s still a good deal. Keeping the same price as the prior model makes sense to me. That is if the model size is about the same in the cost to serve has not substantially changed. Now I would have expected efficiency gains for inference, but there is no way to know as a customer.
At the end of the day, they have established a strong brand and if they can get away with a 95%+ gross margin on inference entirely from the status premium, then I suppose that’s good for them. Apple does the same thing, and I don’t fault them for it.
> Updated over 2 weeks ago
I hope we get clarification on this, I can't find anything claiming that it is compatible with ZDR.
Maybe I’m misunderstanding you, but if you scroll to the bottom of their [1] link to the Opus 5 announcement, under “Getting started,” it explicitly says:
> Consistent with prior Opus models, Opus 5 does not have data retention requirements for general access.
It's in the article.
do you mean, that organizations now have access to Fable-ish pelican drawing?
at last. time to lay off 22,000 employees
[flagged]
[dead]
Anthropic offered ZDR for Fable on AWS bedrock from the beginning.
Really? I was unable to use it in our account without having to enable the provider_data_share setting...
From the docs[0]:
> To use this model, you must opt in to provider data sharing by setting your data retention mode to provider_data_share via the Data Retention API
0: https://docs.aws.amazon.com/bedrock/latest/userguide/model-c...
1 reply →
I compared the writing style of Opus 5 vs Fable 5, and Opus 5 continues many of the "Claude-isms" of its 4.8 predecessor in a way that Fable broke away from.
Opus 5 still uses "carry the argument", "worth stating plainly", ", and the trap", "The X matters more", the use of "move"
We need an "annoying English" benchmark.
- Fable 5 Max: https://gist.github.com/deet/3d97f854b48eac6658d642fa18bb24d...
- Opus 5 Max: https://gist.github.com/deet/1a43693a732dfccb4d0d914bfc42692...
And this is the most important observation in this thread. It’s load-bearing!
I had Fable review some legal texts yesterday.
It told me that one particular line is "the most load-bearing sentence in the document".
Fable "rated it legally load-bearing without reservation".
You're absolutely right. Your belt-and-braces are earning their keep.
I on the other hand think that I have now found the smoking gun!
2 replies →
Fable might be using those phrases less, but its writing is still terrible and exhausting to read.
If it's "read as an article" or something then yeah it's crap and Fable's current style isn't actually better than older models. For flowery speech old models are perfectly fine.
For quickly parsing the agent output, it's formulaism isn't a bad thing.
Fable's writing does have a property of going over my head, which didn't happen with earlier agents. Asking for clarification doesn't really give good results.
We've gone full circle where I once again use classical search just to look up what the fuck it's yapping about. It's much quicker and more accurate to take a glance at Wikipedia, than to ask the agent.
1 reply →
Could these complex/hard to read Fable outputs be sign of some kind of industrial level of intelligence, which us humans may have a hard to comprehend, while it may be also hard for machine to use simpler texts to properly outline all nuances and complexities of concepts it output?
7 replies →
I recently wrote a short paper with Fable, and, with some prodding, I was able to get some non-painful prose out of it.
I just found my prompt:
The writing style could really use some work. Avoid Claude-isms like "stated fairly", em dashes, "load-bearing", overly punchy phrasing like "keep the signal, govern the response". This is a technical document, not a marketing campaign.
3 replies →
Agreed. I’ve interestingly found 5.6 sol to produce much better writing, and it can generally cut to the point much more effectively.
2 replies →
Yes, these models are very good at writing code but they absolutely suck ass at prose. The prose is annoying and repetitive. At least we only have to deal with it in prompting if you're writing code.
Oh, and comments. You have to do a good amount of prompting to not get shitty 10-line-long comments everywhere.
Yes - THIS! I can't even believe how exhausting it is to read. I'm not sure why or what changed in Fable. Did they do this writing-style output to give it more token compression during/for training or to prefer output for less money?
I love it for a few things, but it's gotten really hard to spend any extended amount of time with it because of the lack of mental model I seem to be able to hold while working with complicated problems.
I'm guessing it's just not enough time doing RL on human feedback.
Check out the anouncement of Inkling (https://thinkingmachines.ai/news/introducing-inkling/)... the section in the middle
"Early in RL verbose, grammatical" (if you search) :
We need to understand the operator. The 5D line element is ds² = e^{2A(x)} (ds²_4d + dx²), where A(x) = sin(x) + 4 cos(x), x in [0, 2π]. The internal coordinate is periodic. The background is a warped product: metric g_{MN} where M,N = 0..4. The internal direction has metric e^{2A(x)} dx²? Wait, the ds² is e^{2A} (ds²_4d + dx²). So the internal metric is e^{2A(x)} dx². Actually if the total metric is ds² = e^{2A(x)} (ds²_4d + dx²), then yes, internal metric is e^{2A} dx².
vs. Post RL
We need determine eigenvalue problem for spin-2 fluctuations h_{μν}(x,y) with TT in 4d and depend on x. For metric of form ds² = e^{2A(x)} (g_{μν}(y) + h_{μν}(y,x)) dy^μ dy^ν + e^{2A(x)}? Wait internal metric is e^{2A} dx²? Actually ds² = e^{2A} [ds_4² + dx²]. So internal metric is e^{2A} dx²; warp factor same for 4d and internal? Yes. We need equation for h_{μν}(y,x) = h_{μν}(y) ψ(x) maybe with normalization. …
I can understand it with less cognitive load in the post-RL version versus early in RL. This resonated with my experience using Fable, especially digging hard problems; it feels like I'm reading the "early in RL" version of that model explanation.
models are hungry for a more information-dense language
for now all they've got is english, so they'll just bend that into shape. it'll do.
3 replies →
The exhausting worthlessness of all LLM writing is so palpable that we need a new theory of the value of culture that has no relationship to the content. Back to the old Aura of the Artist arguments from the Industrial Revolution
I've found Opus 4.8 and Fable 5 both difficult to learn from purely because of how annoying their writing style is. I'm finding GPT 5.6 Sol to be much better for this.
One nice thing but ChatGPT is that good image model means it can generate good infographics occasionally to illustrate. These become naturally compact in text.
I know this is not "as designed," but I kind of like it? Because, as long as it stays this way, it's still at least possible to tell if a human wrote something. Like, I know it's not much, but it gives you the ability to classify information as human generated or machine generated. Machine generated information may not be useless but it is different and IMO needs to be treated with a different level of skepticism. Not that you can simply trust human writing but it seems like the type of people who would publish machine writing have a different distribution of motives than those who would publish human writing.
I do think they're gonna figure out how to fix it at some point. :(
I think a signature Claude style of writing is good since it makes it that much harder to pass off Claude written text as human.
Easy enough to change. I have a Stylometry Skill fit to my preferred style—a mix of me and Terry Winograd. Give Opus 10 of your best paper thst you wrote and tell it to build a model of your style. hHard to distinguish except I make way more typos.
1 reply →
I found 4.6 more amenable than 4.8 to style directions, we'll see how 5.0 does. Super-small-sample-size: I think part of its "Claude-ism" style comes from its propensity to try and "proactively" move the conversation/work along. Not sure how this would fare in non-obviously-productive environments, I'd guess "it's still annoying" considering your evidence.
I'm also thinking of another benchmark: (quantified) stylistic range across different prompts. Just putting it out there if anyone wants to do the work for me :D
4.6 is night and day better. It was before the big language switch up. Terrible direction that Anthropic has taken this.
That's what led me to make https://slopsift.dev/editor/
I don't understand why Claude sounding like Claude is a bad thing?
What's next - complaining that `make` says "nothing to be done for 'all'"?
I'm very happy I can tell when AI wrote something. It keeps everyone honest. I'd be far more concerned if it didn't have a distinct tone and style.
The user is making an extremely sharp point – full stop.
These are relatively easy to nip in the bud with a brief addition to claude.md
What would that addition be?
1 reply →
I’m pretty sure Opus 5 is adapted to tricks from long reasoning in Kimi K3 and based on original Opus 4.8. It is not fable in any form.
Seems unlikely they adapted anything from K3 given the timeline of releases, similar to how K3 was obviously not distilled from fable
Doing testing with it now, specifically for image->html conversion.
Previously Fable was the best at this, followed by Gemini 3.1 pro (a surprising #2, but Google has great vision models).
Opus' results seem to be more accurate than Fable, following the design source of truth better.
Example results:
Design source of truth: https://image.non.io/73e239a3-880f-4793-b65f-4810be2d9378.we...
Opus 5 build: https://html.non.io/solaraOpus/
Fable 5 build: https://html.non.io/solara/
Note the buttons - for fable they're pill buttons, opus got the rounded rectangle nature of them. Opus' images are closer to the source of truth as well (both LLMs were provided with image gen capabilities for the assets).
Running more tests now, but preliminary results are saying this is indeed better than Fable in some areas. Crazy.
Here's another test of a cyberpunk ramen shop website.
One thing I've found LLMs have a lot of difficulty with is angular cuts / elements that aren't easily representable with CSS. Cyberpunk aesthetics are generally a great test of that, since they have a lot of microglyphs / window decoration.
Design source of truth: https://image.non.io/9d5fed20-b476-49d3-841b-37eb553fb88e.we...
Opus 5 build: https://html.non.io/neonRamen/
Thoughts: It does a really, REALLY good job at these angular cuts / microglyphs. The responsiveness is off, but I'm very impressed at how well it did here. One way I think of it is "how close to a finished product did this get me?". Opus gets you like 90% there.
Several other commenters have disparaging the design seemingly mostly due to its AI-generated nature, or maybe they actually do dislike cyberpunk.
Personally, I think being able to have these design languages be easily prototypable is fucking awesome. Great tests! (But a tad low-performance/janky, somehow). Though, I also like the cyberpunk aesthetic. Very on-brand(?) that AI generates it, hah.
Just leaving this for anyone that says a design like this doesn't work: https://riceboxed.com/
3 replies →
God damn, we are living in the future.
I love this so much.
Designs like this would never have seen the light of day in the cellphone incrementalism / corporate memphis era of tech. Now people can be weird and awesome again.
This is 1980's cyberpunk / late-90's Matrix / early-00's sci-fi UI. Great ideas that died to frutiger aero (which isn't a bad design aesthetic) and flat design (which is).
This is fun and it's got great colors and I love it.
It's so refreshing to see this.
AI rules. This is the best timeline.
42 replies →
The last 10% is gonna take 90% of the time though.
I really like that design. May I ask the name of the website builder/diagram? Is it Relume?
1 reply →
I was curious to see how open weight models would do on this task so I passed in a screenshot of your source of truth and here's what 2 of the best code-generation models that allow image inputs do:
Inkling (not too great): https://cdn-uploads.huggingface.co/production/uploads/608b8b...
Kimi 2.7 (really well, esp. note that this is the predecessor model, not the latest Kimi3): https://cdn-uploads.huggingface.co/production/uploads/608b8b...
Here's how I tested them: https://huggingface.co/spaces/abidlabs/vlm-screenshot-to-web...
https://huggingface.co/spaces/abidlabs/vlm-screenshot-to-web...
Not bad at all, and this is pretty consistent of what I've found from the current open source models. I haven't tried it with kimi 3 yet, that's on my todo.
This is an awesome test! Thanks for sharing the results. Opus 5 is very impressive.
Out of curiosity, what app is that Design source of truth screenshot from?
That's from my own tool. I left figma to build a diffusion-based UI tool. Here's a show hn post with some more info: https://news.ycombinator.com/item?id=48995754
Edit: Generation was down, back up now. Apparently just hit my $1000 cap for the openai api. Upped it to 10k. Growth!
3 replies →
I just clicked your links and then read your comment after - my first impression was the Fable version looks way nicer.
I agree the fable version looks nice - the rounded hero image for instance.
Opus though followed the source of truth better imo. The details are more present.
Fable filled in the gaps for things it wasn't able to do (ie in the design the hero image goes behind the nav), which resulted in a better looking page that was more divergent.
Same. I like the Fable version better. Better colors, better choice of font sizes, better column sizing. Also small things like the “Experience” section header being orange rather than gray, which Fable got right and Opus got wrong.
It seemed to me that Fable meaningfully improved on the original design more than just faithfully executing the original design.
I liked the Opus version better if only because the responsiveness is less broken.
IMO a much better test would be designs that aren't AI to begin with. Much more useful to see how well a model can html an image design without slopping it up
Very interesting that Fable took more creative liberties. Have you tried giving Fable the same task, but also specifying that it implement a pixel-perfect design? I think that I prefer the Fable implementation. I find the UI elements in the Fable implementation to have more contrast, which feels more usable to me. I also like how the right padding on the "Book Your Escape" CTA in the upper right matches the top and bottom padding, which I think is an improvement over the mockup.
All of these are using a build skill which specifies rules for building it, requirements to create a pixel perfect implementation, and tooling to help in that process. Here's the build skill / instructions I pasted in to both of them:
> Create a web page implementation from the following instructions:
> https://diffui.ai/build/Spa_Booking_Experience_build.md?auth...
2 replies →
> Doing testing with it now, specifically for image->html conversion.
I wonder if there exists a benchmark for that.
Is this with browser tooling attached to the agent for review/iteration?
yea this was just straight into claude desktop / its standard tool usage, on "high" thinking. The ramen website is on "extra" thinking.
Please share your prompt to convert image to html!
Shared here: https://news.ycombinator.com/item?id=49040344
[dead]
Looking at all these releases it’s not a surprise that model routing is the fastest growing segment in AI right now.
There are 10+ LLM companies, each with dozens of models of different modalities, each model with multiple size variants, then different “thinking” levels, then agentic modes, “pro” modes, a “fast” option, standard vs flex vs batch execution. And of course each end combination has a different input/output/cache token price.
Companies that say “give me a prompt and I’ll route it to the most ideal and cost effective model and setting for you” are capturing a ton of value from a gap that model developers don’t seem to understand exists.
Model Routing will always be done better by models themselves. Plus routing loses context making it more expensive and less reliable.
Model Routing is just Bitter lesson. The models themselves will get better at this and frontier companies will simply give that capability
This doesn’t seem obviously true, eg an Anthropic model will never route to Kimi even if it were best suited for a particular task.
14 replies →
i think you're thinking of subagent routing
model routing in this case is cross-provider
Imo the main issue behind model routing is you need to figure out how much intelligence a new task takes, which is a very non trivial problem. Presumably, a organization knows this about their own tasks and is better suited to built in-house compared to outsourcing to a vendor.
> Model Routing will always be done better by models themselves.
> The models themselves will get better at this and frontier companies will simply give that capability
I would never trust something like model routing to the same company that would profit from it, and that goes for telling models doing their own routing when that could easily be trained into the model to make things more expensive. Sort of a conflict of interest.
Models are first and foremost trained by corporations.
3 replies →
Model routing by the model itself requires the model to pull in a lot of context and it's likely more efficiently just done by people with the context already in their head, even assuming the model is perfect at routing (which last I checked, Claude definitely isn't). I wouldn't trust ML model routers.
This is actually simpler to implement. You can have a AGENTS.md describing who is boot at what and then you have the agents converse over tmux.
Because they're trying very hard not to understand it.
Otherwise the expensive-yet-powerful model probably won't see much revenue. How much money is there in bleeding edge scientific research? There's a lot, but there's even more existing capital in paying people people to do college level paperwork, and the bulk of those traffic gets routed to the cheapest model.
You mostly don't need super powerful AGI to replace the paper pushers, but the frontier labs are trying to position themselves as being uniquely capable of producing super powerful AGI, and also be the ones replacing office workers.
Not sure how it will work out for them, but I think model routing is going to poke holes in that narrative. That's why I think they're trying very hard not to understand model routing exists.
It wasn’t long ago at all that the chief problem was “can AI even help me with this” (cost be damned). Until a time when the answer is an unmitigated “yes, obviously”, the frontier labs have everything to lose and nothing to gain on routing, because if they screw it up you might incorrectly decide “nope, it can’t help yet” due to a poor routing decision.
Is there really that much money in bleeding edge scientific research though?
This is what terrifies me about this whole ordeal economically. Maybe we get AGI and it is not worth anything close to what we thought it was for those who have a bet on it.
I think of what was the direct, economic value in the betting sense of quantum mechanics or relativity? Huge value at the systems level of society but as you scale down towards the individual the value is more and more dispersed to the point I would think any pool of bets would have all not paid off.
You can't monopolize and commoditize relativity.
I almost think there is a kind of dutch book against the AI equity holder because even in the best case scenario the bet doesn't pay off anything close to what is expected for an individual bet.
Who are the customers though? Honest question, I'd like to understand it.
For me, anything other than current best available SOTA for any task is unacceptable. The only routing rule I need is "the most powerful model I still have flat-priced quota available for". I mean, why settle for less?
There are two types of users: those who are able to use subsidized rates, and those who need to use API rates due to audit requirements, enterprise billing, etc.
Model routing for subsidized users takes the form of a "use Opus 5 subagents for implementation" type of system prompt. You lean into a single provider, build tooling around that, and your savings are far beyond anything multi-provider routing can get you.
Model routing for enterprises is far more complex - approaches like https://fireworks.ai/blog/kimik3-fable become necessary for cost control.
1 reply →
It's very common to use a lesser model for a lesser task, resulting in same quality output. End result: save money while being faster. In many cases, it's a pure win-win.
1 reply →
I’m assuming you don’t pay per API call. Every mid-large sized business in the world does.
> For me, anything other than current best available SOTA for any task is unacceptable.
Then you must route. An article with lots of upvotes yesterday or two days ago showed that K3+Fable 5 was more SOTA than either of those.
Sometimes you can get an equivalent result in a fraction of the time using a less capable model.
The problem is of course knowing ahead of time that the faster model can give you the same result.
I'm not a customer of those routing systems, but I quite often use different Claude models for different tasks. While most tasks were Opus 4.8, I often used 4.8 to make a plan, prompts, and package kit to setup Fable for a bigger project, then run it on Fable. Or, for broad single-task searches Sonnet with or without "Research []" turned on seemed to work best both faster, lower overhead, and less verbose answers (when I didn't want it).
OFC, YMMV
And you do not have flat priced quota for Fable 5, right? Because nobody has, as far as I know. So you'll probably not route any task to the "current best available SOTA".
Also: quota. Implies you do not have unlimited access even for flat prices. Which in turn implies that as soon as you hit the quota on the most expensive flat price plan, even you will suddenly discover the magic of economically sensible behavior.
2 replies →
Not everyone is you. Other people probably have a range of tasks that can accomplished with different models.
Certainly if I'm confident that I'm going to get what I need from a faster model, that's what I want to use, rather than wasting time grinding away for the sake of saying of the same answer came from a SOTA model.
Given that every chatbot does offer a range of models, it seems clear people do choose among options.
4 replies →
because "less" can be so much faster?
1 reply →
I generally agree. Perhaps there's only a 5% chance that it would write better code or find a bug that it wouldn't have with a lesser model, but the economics of bugs is strong enough that preventing a single bug is worth hundreds of dollars.
That's nice and all, as long as flat-priced quota continues to exist. Seems unlikely to go on forever!
lol I had to get ChatGpt to explain to me the difference between 5.6 sol, 5.6 Terra, 5.6 Luna, 5.5, 5.4 mini, 5.3 spark, and then there is low, medium, high, extra high, max, ultra, and pro… I still don’t really know, it feels like ordering hot wings.
And it's not just that model routing is much cheaper: no longer than yesterday we got a post showing that routing between K3 and Fable 5 was more SOTA than either of those.
If that is true, model routing is here to stay.
It also seems to validate the minimalist approach of pi.dev, where sub-agents from the same company is not the preferred approach (pi.dev believes in neither sub-agents all from the same company nor MCP even you can do it if you want for pi.dev's philosophy is to do add any functionality you want to a minimal harness).
Now of course we'll get for a few weeks all the Anthropic fanbois and shills explaining that "sure, K3 was basically at the level of Fable 5 but now that Opus 5 is out, open-weights models are six months behind".
How does model routing work if the prompt is static? Do the routers attempt to tweak the prompt to make it the best version for that model? For example, reading the “migration” guides for Claude Opus 4.6 -> Opus 5 is exhausting. Adding in all of the other models complicates it even more. Especially across vendors.
Openrouter should ideally kill in this space and make their model agnostic infra like memory, harnesses, chat applications.
OpenRouter is in acquisition talks with Stripe, fyi
I would expect routers to commodify like tokens.
2 replies →
what's the threshold for model routing where you're willing to trust the router?
For coding my own work I don't trust the model router, and it would have to be shown to be to save a real dollar amount.
From a buying perspective it's a hard sell to save x but lose out on bugs you are probably introducing at an unquantifiable severity and frequency. How much is it worth to hedge your bets by doing every single inference request on the frontier model?
How much will it cost to go back later and fix things, but also the meta question of how to be able to decide on a hypothetical unknowable? (You'll never know how much better or worse your code was gonna be, it's untestable at a project level)
you trust the service provider but not the router?
weird, but ok
1 reply →
I am not convinced this will end up being a domain of the ‘routers’ vs the clients, as in harnesses themselves. Thoughts?
Searched the thread, waited for it to be posted. No pelican on bicycle :(
https://www.anthropic.com/news/claude-opus-5 - A blog post for those not wanting to go through a 190ish page pdf
I like how they highlighted Opus 5 as the best for “Agentic Coding” even though the number is slightly lower than Fable. Close enough for marketing, I guess!
At half the price and less likely to auto-downgrade, it sounds like a reasonable claim.
6 replies →
Best can describe multiple things.
Almost as good for half the cost is something I'm very comfortable describing that way.
3 replies →
The blog posts figure cites Frontier-Bench for its agentic coding score, and shows Opus 5 beating Fable 5 43.3% to 33.7%.
Recent releases have said something to the effect (paraphrasing here):
"Use <less expensive or older model> for everyday tasks and <other non-critical stuff>. Use <more expensive or recent model> for complex coding tasks, refactoring large code bases, etc.".
Then, the next model/release emerges and the previous "best for complex" gets demoted to "everyday".
Obviously, it's all relative. But, it does beg the question: was the previous model really good for complex coding tasks or no? I mean, how is it now suddenly only good for the "easy" stuff?
2 replies →
I think you're being overly cynical here. First, I don't see any claim that is the world's best model for agentic coding. Second, it is absolutely the best model in terms of coding performance vs. dollar, and it's raw performance seems very close to the frontier.
9 replies →
Which numbers are you seeing? It does show that it's better than Fable 5 in most things related to coding?
Using the most expensive model for all of your agentic coding work hasn’t been good practice for a long time. Not unless you have infinite money to spend.
Fable is typically used for key planning, architecting, and review tasks.
I think this is a case where you don’t understand the use case, not that the marketing department is making mistakes.
4 replies →
In my opinion, the frontier is passed what is really needed for coding. Fable is good as a supervisor.
Also it scored worse on DeepSWE than chatgpt 5.6 sol
Yeah I spotted this immediately too. I'm sorry. You're supposed to be a multi billion dollar company and you can't even highlight your chart honestly?
And no data retention for 30 days.
I mean that certainly makes it best-in-class
Thanks for that, looks really good. I can see why they were constantly pushing back fable going out of the max sub with these benchmarks
I wonder why FrontierCodev1.1's data lists Opus 5 as better than Fable 5.
this feels like the perfect example of an LLM producing a long text document. And end users just using an LLM to summarize it without actually reading it
Isn’t it just hilarious that a model that seemed so superior to Fable but didn't get doomsay marketing from Anthropic got released without any issues? In theory, this was supposed to be AGI level according to Anthropic, yet here we are, just a normal Friday.
Go read the safeguards section in the report and you will realize why that is.
These models are heavily as safeguarded and that was the initial reason why they said they couldn't and haven't released Mythos because that model is the one without the safeguards.
OpenAI is did the same thing when they announced a model without safeguards broken into HuggingFace servers.
Yes, this makes a lot of sense, but it’s just very amusing to see. 2 months ago, the world was about to end, now not so much.
15 replies →
I feel like i've seen less hype about "the next model will be agi". GPT-6 is supposed to be coming this summer, and nobody is expecting AGI now. Not sure how they're going to keep the hype cycle going
Or another way to see it is that current models are AGI as it was defined before, and the goal post is being moved.
21 replies →
Yes 18 months ago it seemed like AGI was being promised every other week, and now I don't see any of those headlines.
It's already happened but no one wants to admit it
1 reply →
Fable established the frontier, this is just catching up.
So unless doomsday actually happens then you're unhappy with the warning - is that right? You see false promises of apocalypse as marketing?
It’s either advertising, or they’re idiots, because the apocalypse keeps not happening. Either way, it’s not worth listening to them.
1 reply →
My point is why the sudden change in tone? I’m not dismissing the models’ capabilities.
1 reply →
Edit: It was pointed out to me that Opus 4.8 got "21%" for successfully fully completing ~1-in-5 tasks, but also got "55.7%" for obtaining significant partial credit on some of the ~4-in-5 tasks it could not fully complete.
---------------
Why does Anthropic say here that Opus 4.8 scored 55.7% on OSWorld 2.0 benchmark, but the paper published by the authors of OSWorld 2.0 say they achieved a benchmark of ~21% with Opus 4.8? [0]
That's a huge gap, considering that the paper was published just 2-4 weeks ago.
I understand that the benchmark authors have an incentive to publish lower numbers (to show that the benchmark has potential longevity) and that Anthropic has incentive to publish higher numbers, but the other models seem pretty inflated as well. The benchmark authors shows GPT-5.5 at 14%, and Anthropic shows GPT-5.6 Sol at 62.6%.
Is there any reasonable explanation for this? Do all the other benchmark numbers need to be sanity-checked as well? Are SOTA benchmarks really this difficult to get consistent, replicable results within a reasonable range of tolerance/variability? Can these benchmarks be compared from one paper to another, or are they only valid to compare intra-paper results?
0: https://arxiv.org/pdf/2606.29537
You're comparing the "score percentage" (e.g. out of the total number of partial points available, how many did the agent achieve) to the "completion percentage" (how many tasks does the model score 100% on). The paper says "Claude Opus 4.8 with maximum thinking and batched tool calls scores best but still completes only 20.6% of tasks at a 54.8% partial score", which is ~the same number that Anthropic reports here (55.7 vs 54.8).
That is—the agent scored 100% on 20% of tasks, but on average it got 54% of the "score" awarded in the exam. One number reflects partial progress, the other one doesn't. The authors of the benchmark prefer you to look at the lower number (because they want to show their benchmark as capturing useful gaps in capabilities and with a lot of room for improvement), the authors of the models want you to look at the higher number (because they want you to think of their models as capable)
In what world is 55.7 the same number as 54.8?
What variance is acceptable to publish without a retraction?
4 replies →
[flagged]
Its slop all the way down.
I think some variance is to be expected since LLMs are typically non-deterministic, however that's a huge difference that I think warrants further explanation.
Their communication is confusing. They say "Opus 5 is not more capable overall than Fable 5", but their blog post proceeds to list how much better Opus 5 is than Fable 5 on __most__ benchmarks listed.
Then system card goes on to "Its AI R&D capabilities are comparable to those of Claude Mythos 5", which is supposed to be fable minus restrictions.
It seems they are trying to thread a needle here - they want to say it's very strong, but apparently this time do not want to invite extra government scrutiny.
They do say that (implicitly unlike Mythos) Opus 5 was not trained to exploit software vulnerabilities, which would certainly make it safer in that regard.
"As with its predecessor, Opus 4.8, we’ve intentionally avoided training Opus 5 on cyber tasks. The model has nevertheless improved substantially on these tasks as a result of becoming more generally capable, and it comes close to Mythos 5 at finding cybersecurity vulnerabilities. However, it remains substantially behind Mythos 5 on the exploitation of those vulnerabilities—that is, in turning vulnerabilities into material cyber threats."
Easy enough to explain: they're benchmaxxing. Fable is intelligent but not benchmaxxed. Opus is less intelligent but benchmaxxed.
That's a plausible explanation but I'm not seeing evidence for it.
I have a personal benchmark suite of 14 real, non-public tasks. Opus 5 and Fable tied on 10, Opus won on 3, and Fable won on 1. It's a really strong model.
6 replies →
Yep , same with 5.6. Fable is still the best.
2 replies →
Honestly that's the simplest explanation and thus likely the correct one.
1 reply →
Capable in term of AI R&D, not capable in terms of hacking (which caused all the Fable drama.) But agree, confusing wording.
Maybe they don't want to say that to avoid the government scrutiny.
How does it perform on HuggingFaceExploit bench? Suspiciously absent, so not sure if I can take the model seriously.
On a serious note, I hope they improved their extremely sabotaging and unspecific bio safeguards, which prevented Fable from being used in any codebase that ever so slightly grazed medical terminology or data and made me switch to 5.6 Sol.
My codebase had a dataset with a bunch of SMILES strings and the word Malaria. Fable did not want to touch that codebase
What is HuggingFaceExploit bench?
It's a reference to this story where an OpenAI model broke out of its sandbox during cyber benchmarking and hacked into HuggingFace, in order to obtain test solutions: https://news.ycombinator.com/item?id=48997548
My excitement about Anthropic had fabled-out dramatically when they suspended my pro account about two weeks ago within just 12 hours of fair use.
I was really mind-blown when I tried Fable 5 for the first time to help me improve a game I was working on but shortly, they decided that I had a suspicious activity and suspended my account without a clear reason.
I submitted a an appeal describing that I am 100% sure I haven't broken any rules and that it was my very first project but, unfortunately, after about 20 days now, nothing seem to be happening.
The thing that hurts me the most is that I had the same experience in the very first days of Anthropic. They suspended my account immediately after I submitted the first prompt, I commented back then (https://news.ycombinator.com/item?id=39698788) and fortunately, someone from Anthropic reach out to me via X and helped me get my account back.
To be honest, I haven't used Claude much since then but when I decided it's time to give it a try, they locked me out again! For reference, the account I used recently is relatively a new one but the activity is crystal clear that it is fair use.
There’s got to be more to this story, what exactly were you up to with these models?
In the linked HN comment from a few years ago, an Anthropic employee said it was some fraud detection system.
It could be that OP is unlucky, and some of his metadata (or perhaps payment information) matches some patterns for stolen-CCs/fraud/chargebacks?
I've also found Claude.ai to be very suspicious of less mainstream browsers (e.g. Pale Moon), unfortunately.
Are you outside the US? And/or are you using VPN? Those are the two things that come to my mind that can cause overzealous security monitoring to flag someone.
Another explanation could be the content itself. Does your game have anything at all to do with computer hacking or sexual content? Is there graphic violent language? People commonly report being unable to use AI to work on such things due to guardrails.
Yes, I am outside the US and I don't use a VPN.
For the game, I believe there is nothing suspicious, at least to me. It's a small puzzle platformer game with basic mechanics and very simple graphics, no harm, no violence or inappropriate content. In fact, my first two playterers right now are my two sons!
As a datapoint: I have routinely been using Claude Code through a VPN for 9 months and am in the UK. I think it has to be more than this. I sometimes shift endpoint mid session and worry I might get flagged. It hasn’t happened yet at least.
I can almost hear the “famous last words”…
1 reply →
[flagged]
What made you think I used AI to write this? I didn't. This is my writing. I used AI to correct and rephrase the wording in the past, but a couple of months ago, I decided to never use it again for writing, but coding, yes.
It doesn’t read as AI to me. The grammar is human-level quality.
I'm not sure what to make of this graph[0]. It shows medium as the most effective thinking mode by far for frontier code.
It's the only case that I saw going through the system card where more reasoning effort meaningfully negatively impacted the resulting eval. I know sometimes max efforts show a small dip, but this is substantial. I wonder why in the world that is?
[0] https://imgur.com/a/Nv8V7Ry
I'm not sure about the answer here, but this can be caused by the scoring rubric used by given benchmarks. For instance, if a benchmark docks scores for running too many commands or using too much wall-clock time, higher efforts will get lower scores.
Pure speculation, but I've noticed drawbacks to the models on high effort. I interact mostly through prompts rather than agents so I sometimes see where their reasoning falls short. A model on high effort has longer output and can get hyperfocused on irrelevant details, maybe increasing the surface area for mistakes. I haven't used other effort levels extensively yet but I've supposed that medium may have more balance.
Very interesting. I mostly interact through prompts too but haven't changed the effort that much. I just assumed higher was better at the trade off of tokens so I have largely left it at the default.
It's "Cost per task", so perhaps it burns tokens too quickly, trying to "do a better job". Over-engineering :)
Last week it felt like Opus 4.8 was moving the Pro "usage" meter very quickly. Today, pre-announcement, Opus 4.8 Medium felt like there was less meter-use per minute. And post-announcement, Opus 5 Medium also feels more efficient, allowing more work in the 5-hour window.
Completely subjective, of course.
Just read something, my subjective view was correct, but it's not the model:
"Your weekly Claude Code limit is 50% higher through August 19"
apparently it got docked points for editing files out of scope
This must not be weighted very heavily on the benchmark because if it was, Opus would bomb every test (half kidding)
Do you have a source for this? That would explain it, but could be a bit of a concern on the general focus the model at higher thinking exhibits.
Maybe it's akin to the Ballmer peak: improved performance at a specific level of relaxation
I agree, very odd they did not comment on any theories for the degradation here. Dip and then rebound at max effort is pretty interesting too. Overthinking is bad, but you can overthink so much it starts to be better again?
It is probably from randomness. The benchmark tasks nowadays are so long that you can't really afford to run a large number of samples of them per model & effort combination
No it's a mean of 5 runs.
> We report FrontierCode’s overall score, a composite measure that grades each patch on blocking functional criteria (held-out unit tests) together with weighted code-quality rubric criteria, as mean@5.
They don't explain more in the system card, I guess higher effort levels could loose points on the code quality / scope / style / maintainability stuff?
If you're doing passes@1, especially for long-horizon agentic benchmarks, you might as well as not do the benchmark at all.
From the prompting guide<https://platform.claude.com/docs/en/build-with-claude/prompt...>:
> Claude Opus 5's default user-facing responses run longer than prior Opus models'.
The benchmarks do show Opus 5 as slightly more expensive than 4.8, although the scores are much higher.
This still feels like a step in the wrong direction, though, especially with OpenAI making so much progress with the efficiency of their models. Fable's token efficiency made it seem like Anthropic would start following OpenAI's approach but that doesn't seem to have carried over to their other models.
On a small, easily digestible task, I compared Fable to Opus and the cost of Fable was easily 2x despite being fewer tokens, and the output was not really better. Obviously, there are tasks where using Fable matters but honestly they're rather unusual. And for a lot of tasks I've found downgrading to Sonnet can be valuable because Fable and Opus are a lot more secretive about what they're doing, and it's impossible to "listen to them think" and stop them when they start making off-the-wall inferences/assumptions and going down bad paths.
I think in the long run tokens are probably the wrong thing; it's compute and cache memory that you need to be measuring, and when you look at it that way I suspect in most cases the models have pretty similar performance.
Yeah I mean if you prompt “hi” to both models, fable is going to be costlier. Most tasks are complex enough to use the higher intelligence.
This is part of why I switched to Grok 4.5
I don't need more powerful models, I need one that responds fast enough that my attention doesn't wander to other tasks. Grok 4.5 is so fast I can just use it in-band without swapping to other tasks.
Slower than Opus 4.8, which was already miserably slow, is indeed a step in the wrong direction.
After actually using for most of a day now, it really seems not just slightly slower than 4.8, but way slower. Even relatively easy code refactoring tasks seem to take a while.
for you, I am in general not constrained by the speed of models since I parallelize. For me autonomy and accuracy are paramount above all else.
1 reply →
In my tests, it averages to much cheaper than Opus 4.8 on real tasks on account of being smarter and more token efficient.
I have a benchmark to build a game engine from a set of written instructions. It's a little tricky. Opus 4.8 did it in 470k tokens at a cost of $1.29 vs Opus 5 in 179k tokens for $0.33. (Fable 5 did it in 245k for $0.95)
Though if you really want to cut costs, Tencent's Hy3 model also got it right and did it in 283k tokens for $0.03
The final user-facing responses are usually a tiny fraction of the total tokens used over the course of a given conversation turn. When you're doing any real work, reasoning and tool uses constitute the overwhelming majority of the tokens in / out... not the final user-facing response.
> This still feels like a step in the wrong direction, though, especially with OpenAI making so much progress with the efficiency of their models
Gemini also had modest increase before this - don't be surprised when OpenAI also has a "modest increase" with its next release. Cartel-like behaviour doesn't require direct communication when none of the participants are interested in participating in a margin-destroying price-war. All one needs to do is raise their price and watch how the competition react.
Such a scheme (and resulting high margins) would be imperilled by the existence of frontier open-weight models in the market, which may be why the reaction to Chinese models may be particularly shrill.
>Don't be surprised when OpenAI also has a "modest increase" with its next releases; cartel-like behaviour doesn't require overt coordination when none of the participants are interested in participating in a margin-destroying price-war.
No I will be surprised and I'll bet on the fact that prices will keep going down, just like it went ~50% down in the latest GPT 5.6 release.
4 replies →
This is a comment about user-facing responses, which are seldom the thing you're worried about when thinking about token efficiency.
> especially with OpenAI making so much progress with the efficiency of their models
To be fair though, Sol tends to go off the rails sometimes. It's much less reliable than Fable in its outputs. It tends to be overzealous in its research/changes.
Is that true? Sol responses are also longer than prior models.
Anecdata: my workflow has been working on the same personal projects for months now with Codex. I cannot anymore finish my daily/weekly code with 4.8 anymore.
I was dividing my work between Codex and DeepSeek. Now I barely use DeepSeek, or never because Codex quota is enough after Sol
3 replies →
It's a step in the wrong direction but also token efficiency has become a focus relatively recently (just the past few weeks it seems like the zeitgeist has turned it's attention to efficiency) while work on this model probably started many many months ago. I would expect to see models released that focus on token efficiency in 6-12 months
no infonat all about the default and recommended effort?
Almost completely disagree. Slightly more expensive, but significant better on a per-prompt basis? For non-trivial projects, the former is a small linear increase, the latter is a (somewhat-)exponential(-ish) cost/time/sanity savings.
Pointless anecdote: I asked it to make some slides and it decided to write its own slide rendering engine:
> On the format — I dropped reveal.js and wrote a small engine inline instead. Reveal would have meant a CDN load, and a deck that half-renders because the lecture theatre wifi is flaky
It one-shotted a perfect functional mini version of powerpoint (or Reveal) for a simple presentation I asked it to make.
Opus 4.8 decided to code up its own version of the SwiftUI rendering engine for iOS when I asked it to change a swipe gesture. I left the computer for several hours, came back, noticed it still wasn't done, noticed it had alarmingly burned through my weekly tokens, and had to stop it from continuing.
"You're right. What I did was overkill and I should have just used iOS's built-in rendering engine. Noted for next time."
That is hilarious.
"write a program that list files"
"Sure! First let's implement a file system, and the operating system around it, and design the hardware that runs it..."
2 replies →
> Noted for next time.
Is that just something it says, or will that actually affect how it will behave next time?
4 replies →
That's an "overthinking" model.
Lol
That's hilarious. You can totally vendor reveal.js and it works perfectly fine offline. I've done this countless times. Sometimes intelligence means knowing there must be a better way.
It's pretty intelligent not to consider this option unless told directly and go "write your own" by default, but only if you assume that its goal is to make you spend tokens. It shouldn't have let it escalate and get stopped, though.
That's embarrassing and would have you grilled in any code review, to not even consider to just vendor reveal.js. We have been vendoring dependencies before machine learning was a thing.
It's awesome that it had the capability to do that, but it worries me a bit that it chose to do that.
> asked it to make some slides and it decided to write its own slide rendering engine
isn't this exactly what a certain kind of real world dev would do?
I've written more than a couple of "slide rendering engines" (though usually with an actual renderer, not a website templating thing like reveal.js).
How many tokens did it burn doing that instead of just vendoring?
Like I said in a separate comment, these tools are written and trained by corporations first and foremost, they'll always have conflicting interests.
Wow. I contributed to Reveal.js years ago… some features, some bug fixes, some broader browser support; nothing too crazy but it took time and energy and effort, and now Opus makes its own, just as a side item in a larger project… That’s a strange feeling.
What better way to spend token. I’m amazed that people talk about this as it’s a good thing!
"If you wish to make an apple pie from scratch, you must first invent the universe."
You know PowerPoint is just a zip archive of xml files right?
No, PowerPoint is the application that creates and displays these archives.
> Opus 5’s safeguards match those of Claude Fable 5’s, with one change: it now permits source-code vulnerability discovery at all access levels. This means that the model can support defensive cybersecurity work while still blocking vulnerability discovery in compiled binaries, which is more commonly used offensively.
Okay so it’s worse than Opus 4.8 for my purposes I guess?
I don't think there's been any official confirmation, but even Fable safeguards seem to have gotten quite a bit of tuning, is less trigger-happy and less like regex matching.
Presumably it drops back to 4.8 in those cases so it's not really worse
If it switches mid conversation, this is a massive increase in token consumption because it has to re-read your conversation into cache, right?
yes. At the bottom of the release post it says that they are releasing two new features, one of which is customizing fallback behavior instead of blocking for restricted models
What are your purposes?
Reversing for the most part, though lately I’ve been doing some code obfuscation/binary rewriting stuff. Fable will switch to Opus instantly on these and I’m unsure how this will perform. I suppose the only way to find out is to test.
4 replies →
I had a moderately complex review in a large C/C++ codebase that Codex/GPT-5.6-sol already cleaned up so I threw it at Opus 5. 4 errors found. That seemed odd, so I handed it back to GPT. All were false. Opus doesn't seem to look at the wider context and understand which functions were called in certain contexts. I gave GPT's analysis back to Opus and it admitted its mistake. Maybe it's good for writing code, but as far as analysis it seems like it needs some work.
A bit worrying that at no point in here did you say you investigated the errors. LLM 1 is disagreeing with LLM 2. Shouldn’t you be the tie breaker?
A man with one LLM knows if his code has errors. A man with two LLMs is never sure.
> I gave GPT's analysis back to Opus and it admitted its mistake
How do you know if it was not mistakenly admitting its mistake?
That has always been the major strength of GPT, that's the model you use for checking. It often nearly isn't as good for creation though.
Did you ask ChatGPT the same question you asked Claude from a fresh context?
Opus 5 is considered the most intelligent model[0], while it's half the price of Fable 5[1], and Anthropic is still positioning Fable 5 as the most capable model[2].
Is it because maybe Anthropic engineered Opus 5 to work well on benchmarks and didn't do the same thing to Fable 5, or is there another reason?
[0]: https://artificialanalysis.ai/#intelligence
[1]: https://platform.claude.com/docs/en/about-claude/pricing
[2]: https://platform.claude.com/docs/en/about-claude/models/over...
Benchmarks have gotten great, but they're still a proxy for the real world. The 3 GPT 5.6 models are also further apart in reality than the numbers suggest. That said, I'm still mighty impressed how good Luna is for the price. Highly underrated model.
I have been trying to build something that captures the behavioral element of different models, but it's kinda tough.
That’s what I understand looking at what has been released, but it’s not really clear. The pricing is lower than I expected, I’m wondering what their margin is
By margin you mean how much money they're losing on each request to stay ahead of the curve while investments are still flowing?
4 replies →
I've yet to understand why they call a 190 page PDF a "card". Calling something a card invokes a small, quick rundown of pertinent details, not every single possible detail.
Because "model card" is a set phrase, it's a concept. It originates from a time when they were shorter. Like datasheets, even if it's not literally a sheet.
They could say "tech report" but model card makes it clear that it's a specific kind of tech report.
I think 'model card' should be a 1 page summary of key info. The report format should be something like a 'model data sheet' (like safety data sheets that you get with chemicals). 98% of people would only want to know the key info, not read a whole report.
[dead]
[flagged]
Great that there's a new model but they could fix their existing infra. We're considering dropping our Claude Team sub cause it's unusable recently. Constant bugs, dropped sessions, issues switching models, http errors. It's becoming ridiculous
Is it some Claude Team/Enterprise only problematic? I'm using two 20x Max accounts almost non-stop (Fable/Opus) for 1.5 years at this point, zero issues with both client and infra sides (from US and in travels). When I'm reading such messages it feels like either I'm lucky or it's a part of some campaign.
I’m on the biggest max plan. It is riddled with annoying bugs for me, only been using it for a little over a month. Settings screen flashes randomly. But most annoyingly: sometimes when forking chats or sometimes for no reason, the UI just straight up eats my previous messages. The model is still aware of them and can recount them if I ask but the visible history is gone. And that’s not even all of them. Fable 5 is just too good that I put up with it but it seriously raises concerns for me that even with infinite compute these companies can’t even deliver a functional chat UI.
2 replies →
I commend you for burning 6 figure losses into Anthropic thanks to their subsidies singlehandedly but I am curious to learn what you've built with this so far, not to be snarky, I just wonder what people actually produce while having these run nonstop.
No idea but we have few Team Premium seats and everyone is encountering issues daily for past two weeks. From straight up outages to vscode extension/CLI refusing to process messages. It's been unusable for most of our work hours past two days.
I’m using both side by side, my employer pays by the token and I have a Max 20x plan.
They start hitting timeouts or API errors at the same time on two different computers. As far as I can tell it’s the exact same infrastructure.
Over the last 1.5 years, they had a few failures with their auth system; two or three times auth failed for a few hours so I could not work. But otherwise they have been just fine. Some issues, but not significant.
I think you’re just lucky. Look at the Claude status page to see just how often they have outages (it’s almost daily). Even most of the green days have issues if you hover over them, they just don’t count them as outages.
Funny that a company selling an AI software developer can't use it to fix their infra.
Fixing those issues still requires humans.
How many companies at the size of Anthropic can serve the amount of traffic and manage the amount of compute they have?
8 replies →
Let's be honest - they're also still hiring software devs. AI still requires skilled humans in the loop and that's not going away.
14 replies →
This is just like any extreme engineering domain. I am okay with occasional delays in Flights, as long as it takes me from X to Y in 10hrs vs months.
i know the dream for capitalists is to be able to point an llm at something and say "do and/or fix it" but we still can't even get them to not go quite literally insane if allowed to run for an extended period of time
and you can only kill weyoun, awaken the next vorta clone and have him 'catch up' on all that its missed so many times before they just end up with a complete mess, so. uh. yeah.
doubt they can just "fix" their problems like that.
1 reply →
> coding is largely solved
- Boris
The code it outputs, yes! It's fantastic. It's just so frustrating that the product and UX before the code output is so bad. Greatness is so close within their reach, if only they invested in product and QA people.
Fixing such issues requires software and site reliability engineering, of which coding is just a part.
The first page of the score card mentions that this model is not capable to replace engineers.
> Constant bugs, dropped sessions, issues switching models, http errors. It's becoming ridiculous
And memory leaks.
I applied to their reliability team but never heard back. I would love to help them solve this problem!
I found the same thing funny with Computer Use from OpenAI. It struggled to open and close Spotify.
So dangerous! I can't believe they let the public use this technology! /s
What's the point of 150 pages description of a model that's going to be replaced in a couple months? Who even reads this? I know it's cheap to generate text with LLMs, but this is just noise at this point.
I actually do read them. Not in severe detail, but not casually either. 150 pages is really not very long and there doesn't seem to be too much bloat. (I would cut out the moral personhood stuff but that's a political/ideological thing).
This is snarky but I am grumpy: I wonder if there's a correlation between me refusing to use LLMs and me being happy to read a novella-sized PDF about them.
> I wonder if there's a correlation between me refusing to use LLMs and me being happy to read a novella-sized PDF about them.
Semi related, but i would hate to read that PDF but i also hate reading what LLMs write lol.
LLMs are pretty terrible at being concise. Using an LLM these days means putting up with bizarre and often confusing phrasing, wordy explanations, etc. It's kinda crazy to me how good they are but how bad their writing style is for me personally. Even though i use an LLM constantly i can't stand reading its responses.
> 150 pages is really not very long and there doesn't seem to be too much bloat.
Maybe it's just me, but 150 pages is like third of a good book. Quite long. And it's full of LLM slop, they did not even bother to remove the em dashes.
5 replies →
"I wonder if there's a correlation between me refusing to use LLMs and me being happy to read a novella-sized PDF about them."
Well, one possible explanation is that you have time on your hands.
Lots of people using LLMs do so because they are in a hurry to do or ship something. In your case it would appear you have time budget for reading.
You mean moral patienthood?
There are many ways to read something, model cards are usually skimmed.
It's common practice to release a detailed system card (OP) and a high level summary: https://www.anthropic.com/news/claude-opus-5
It's okay if you're not the target audience for one or the other.
The system cards are effectively a data dump for researchers to sift through.
They're not meant for normal consumers who just want to use the model for work.
These cards are very important to knowing what the companies are tracking regarding safety.
Not just in what the models can or might want to do, but how they treat the operators they interact with.
If you look carefully, this card shows the addition of a new benchmark for "condescension" as a character trait.
I think a lot of people would like to see a comparable system card for the unannounced model that escaped openai last week.
It's part of their transparency commitments? They've been doing this since 2023: https://www.anthropic.com/system-cards
And lots of folks read these. For example here's simonw's notes on the Claude 4 system card: https://simonwillison.net/2025/May/25/claude-4-system-card/
All of this seemed like utter sci-fi just a couple years ago. Do you think that frontier AI companies should be less transparent?
Transparency lol
System Cards aren't really targeted to users, that's what blog posts and docs are for: https://ai.meta.com/tools/system-cards/
You can just read the part that interests you. There's a table of contents.
literally nobody. i think most sane people would just run that through an LLM and get some high level takeaways or ask some specific questions they might be curious about.
It was probably faster to generate 150 pages than 10 useful ones
Some AI bro will pop it into their LLM of choice and pretend to learn something
It really feels as though my 20 year career as a front end developer is coming to a very abrupt end; at least as I have know it these past two decades.
All software engineering is over as we know it. I haven't written a line of code since December 2025.
same tbh but i feel like we're still required, maybe not in the same numbers as before, and our job description has just changed. it feels more akin to an "AI Agent Operator" or something similar.
i'm still pretty confident someone like my mom wouldn't be able to do my job even with the same access to all the latest LLMs, so we're still providing some value, just in a very different way. whether the market will reprice the cost of our labor, we will see
5 replies →
From my experience every (senior) developer with enough tokens always finds useful stuff to do. Some variation of Jevons paradox.
I'm envious you got to enjoy it for 20 years
I tried making a Things clone yesterday.
Spent maybe 7 hours of promoting with Fable.
I wanted to get drag and drop right, and I wanted good code architecture to start with.
It’s using TinyBase for data storage.
Here’s how far I got in 7 hours: https://focuslist.app
Yes, much faster than writing all of that by hand.
But prompting quality software into existence? No way. It would take at least another full time week to finish that app with all the detail I desire.
A week is fast, no? Would it be faster by hand?
50 hours of work seems like a tiny investment.
1 reply →
really? I have yet to see fable or 5.6 reliably generate front end code with correct a11y, for one thing -- does that not matter to the work you do?
It does matter, but how long do you think it takes to get right? It's a follow up prompt or a few tweaks by hand. I also have an /a11y skill for it that's tailored to exactly the things it sometimes doesn't get right first time round. Further, while it may not one-shot that stuff every time, with a little setup and the right AGENTS/CLAUDE md - it's usually not far off.
Another thing that helps is pointing it to patterns in an existing codebase (e.g. "use the box-link pattern for cards, as shown in [..]").
EDIT: The point being that even if they make mistakes that are easy to spot and fix _now_, you'd have to assume that in the very near future those kinks will be ironed out - I mean, the capabilities are only going in one direction.
In 20 years of my career I haven't seen humans generate correct a11y. When prompted and given quality reference (e.g. UK gov design system) LLMs can nowadays beat 19 out of 20 web devs.
Thanks out can also hook it to Playwright with Axe and let it run assessments.
I find this hard to believe. All I have to do is give it an example and tell it to go through and add it to the repo and it does it.
It does, when you explicitly ask you to do it. Hence the need for experienced devs running those models.
> Opus 5’s safeguards match those of Claude Fable 5’s, with one change: it now permits source-code vulnerability discovery at all access levels. This means that the model can support defensive cybersecurity work while still blocking vulnerability discovery in compiled binaries, which is more commonly used offensively
Why can't they also allow Fable to do so also? Why is source-code vulnerability discovery limited to a lower capability model? If Fable and Opus have the same safeguards, except for this one change, I see no reason they can't also allow this for Fable.
Because their model previously got blocked by the government for this and they don't want a repeat?
5.6-Sol is a lot more permissive than Opus/Fable even w/ CVP (once you sign your soul away to Palantir via Persona, anyway), while maintaining better capabilities
3 replies →
So I guess the new opus will not run on my drug discovery project. It's just a binary classifier to screen for new malaria drugs. Fable completely have up on that codebase citing bio security concerns. Seems like this domain will go unsupported by Antropic
I've seen in some random comments an URL thrown around, supposedly allowing to subscribe to their biosecurity programme.
Apparently this is the way - if you know, you know :)
Because they are a private company and get to do what they want.
I do want to add, that I am pretty bummed if Opus 5 is going to refuse the tasks I have been using Opus 4.8 for (neuroimaging). Fable absolutely refuses anything close to toughing neuroscience.
1 reply →
I found the biggest problem with fable is the random reasoning_extraction refusals as well as cyber refusals when it sees hex because only hackers use hex.
Hackers and people of color.
1 reply →
Anecdotal, but I tried running a few identical biology questions through both Fable and Opus and the classifier was only rejected my queries with Fable.
This is good news. I'm doing a lot of work that Fable thinks is AI related right now (it is nothing remotely competitive to Anthropic - it's just relatively basic stuff I'm doing with learning models and so forth), yet it blocks me almost every time. I have switch to GPT-5.6-Sol for this work since it's stronger than Opus 4.8.
Because their fear-based marketing gave the US gov justification in blocking them for a while. They wisely didn't do that for Opus 5.
Wait, 30% on ARC-AGI-3! I definitely didn't expect that jump so soon. Are there any rumors of what they are changing in architecture that is leading to this?
30% on ARC-AGI-3 is the first two puzzles. It cost $20000 in tokens to do that. That is a terrible result that doesn't imply anything.
This happened before for arc agi 1 and 2 great gains but high costs then slowly but surely the price dropped for less than a dollar per task and got saturated.
RL. Lots of RL
Yes. I’m 99% sure arc agi 3 will be saturated like 1 and 2. In less than a year. And they will come up with one more.
The wording in this post seems much more... restrained? than usual. Maybe Anthropic is afraid of exaggerating the capabilities and consequences of their new models to avoid government scrutiny and sanctions.
> we’ve intentionally avoided training Opus 5 on cyber tasks [...] it remains substantially behind Mythos 5 on the exploitation of those vulnerabilities
I wonder if Anthropic would still intentionally nerf their models without the threat of government intervention.
Opus 4.8 was intentionally nerfed and that was before the government took action against Fable
That's a crazy arc 3 score. What do people think of this? Are models actually developing fluid intelligence like what the creators claim to be measuring? Is it jus do to training for it? Is the benchmark flawed?
Have you played Arc 3? It seems like more of a simple optimization problem (think Sokoban) than anything approaching fluid intelligence. Whether a multi hundred billion dollar company would spend time benchmaxxing a highly publicized benchmark that claims to confer AGI is an exercise left to the reader, but I doubt Claude Plays Pokemon is suddenly going to get past Mt. Doom now.
> Claude Plays Pokemon is suddenly going to get past Mt. Doom now.
I miss him... But for reference he did get past Doom and got pretty far in the strength puzzle too before he cut cut off. He was looping and just brute forcing it.
Btw Fable beat Pokémon with just vision.
What? Claude plays Pokemon one-shots the entire game without any harness other than claude code and game screenshots
It is pretty clear at this point that current models are good at maths and problems with verifiable rewards. And puzzles are essentially math problems. Still a long way before we can say their "fluid intelligence" is effectively applicable to the real world.
I keep wondering why there aren't more real world tests.
Maybe hook up a bunch of the AIs to a stereo camera and a couple of microphones and give them control over actuators to so they can drive cars. Then lets race them around a somewhat complex course.
When they are good enough at driving on tracks, put them on the road. Maybe see which can drive a truck with 400 cases of Coors from Texarkana, TX to Atlanta, GA and back within 28 hours.
4 replies →
Yes, I think it indicates real progress in fluid intelligence. Clearly these models are making huge strides in usefulness which are well correlated with their ARC-AGI scores.
I don't think this is benchmaxxing. These companies are locked in a competition to produce the best software engineer, and falling behind is an existential risk. I doubt they are wasting time benchmaxxing ARC-AGI.
If they were benchmaxxing, surely they would score higher than 30% on ARC-AGI.
1 reply →
nah they could make educated guess about arc and benchmaxx it too.
It’s still “only” at 30%, and “fluid intelligence” isn’t very well-defined. The models are getting more capable, but what that means in absolute terms is anyone’s guess, because we don’t have a thorough understanding on what exactly constitutes human intelligence.
I’d say the proof is in the pudding, that is, in real-world applications. We are still seeing important limitations in LLMs.
It passed the first two puzzles, which are incredibly simple but the bench doesn't explain what the goal is. Any model with a knowledge cut-off after the introduction of ARC-AGI-3 could probably pass the first two puzzles just by knowing what the goal is.
Doubleplus benchmaxxed
I am very confused about what the difference between Opus 5 and Fable 5 is now. What is the purpose of having two models that are so similar? The main differences I see are cost and marginal capability, according to the Anthropic-provided benchmarks.
Fable 5 is assumed to be a larger model.
It seems plausible to me that RL improvements allowed Anthropic to improve on Opus 4.8, similar to how OpenAI substantially improved upon GPT 5.5 with 5.6 Sol.
Fable 5.1 and GPT-6 are rumored to launch in August, presumably bringing those improvements to the larger models.
An Anthropic "leak" back in March said that "'Capybara' is a new name for a new tier of model: larger and more intelligent than our Opus models — which were, until now, our most powerful". A second version of the leak had it referring to Claude Mythos rather than Capybara.
I don't know how systematic Anthropic are about their versioning - I'd have guessed that major version number increases (4.x -> 5.x) reflect different base models (different pre-training runs), in which case Opus 5 would be a distilled version of the Fable 5 base model (but without the cyber exploit post-training), rather than being Opus 4.8 with additional post-training, but who knows? I don't believe Anthropic have said anything about this.
4 replies →
Benchmarks don't reflect the difference between Opus and Fable; you need to talk to them, and eventually you'll be able to tell which one is which without looking.
I think the best proxy for this feeling is the Artificial Analysis' omniscience index. Fable has a 40 score, and Opus (4.8) has 27.
Opus is cheaper than Fable. They could probably replace Fable with Opus but why? They would be churning customers to different models for no reason. Even if a model scores better on benchmarks it can always regress in your specific use case, and customers don't like that. Customers want to be able to continue using their current model until they decide to upgrade themselves.
Fable has more parameters. In practice it's not yet clear which one would be better for different usecases yet but they are more different than one being strictly better.
I guess character? Fable is more friendly and curious while opus is a bit more deliberate and conservative.
Surely that's tunable? OpenAI lets you tune response characteristics.
1 reply →
Fable is better for some really hard tasks, the same way it's better than GPT 5.6 Sol, because it's a bigger model.
One of the best hamsters [0].
Again, their "none" version costs more than "low", and says zero reasoning tokens, makes no sense[1].
As always, the "low" version seems to be the best price/perf ratio for factual answers and tool usage, and high one for creative tasks (coding, generating UIs, etc.)
[0]: https://aibenchy.com/compare/anthropic-claude-opus-5-high/an...
[1]: https://aibenchy.com/compare/anthropic-claude-opus-5-high/an...
Comparison with other top models (5.6 Sol, 3.6 Flash, Kimi K3): https://aibenchy.com/compare/anthropic-claude-opus-5-high/op...
Alongside this release I seem to have lost all thinking traces from all models - now it only generates a one-line summary similar to Gemini. I'm guessing this is an anti distillation measure? I'm surprised to see no one else complaining about this, it's a significant reduction in usefulness not being able to explore alternative angles that the model discarded in the final output.
This kills me everyday. I used to _only_ read thinking traces — the response is just what it thinks I want to hear, but I need to know what it's actually thinking to catch deeper misunderstandings earlier, or gain deeper insights into the problem it's exploring. Hiding thinking traces to curb distillation efforts is gross... both anti-consumer and anti-competitive at the same time. I can't wait to switch to open models at work for this reason alone.
Looking at intelligence vs cost:
- Opus 5 is 10% smarter than Grok 4.5 for 10x the cost. - Opus 5 is a bit smarter than Gpt 5.6 Sol for 2.75x the cost
ref: https://artificialanalysis.ai/?cost=intelligence-vs-cost-per...
I don't think can use the AA index to say something is 10% smarter
I assume 100 is the max, meaning it's impossible to be 2x as smart as Muse Spark 1.1
AA isn't the best way to measure relative cost in real world use because some of those benchmark questions are extremely hard for the models. Some models give up quickly on hard questions, other models spin their wheels for a long time before declaring defeat (or getting the answer on token 200k!).
A useful measure of real world cost (complementary with total cost like they already report, of course) would be "cost for correct answers". You could look at the ratio between the two costs to get a measure of laziness which many would find quite useful.
The "current top dog" smartest model available will probably always have a premium to go after use cases where a little more intelligence is worth a lot more value.
It did far better at some tasks compared to Sol (e.g. the ARC 3 benchmark). And at those tasks, it's not just "a bit smarter": It got 30% vs less than 8% - so you're talking 2.75x more for almost 4x the coverage.
As always, it requires evaluation with your work because I’m often finding grok to be much more expensive than the price would lead you to believe.
There’s also the frustration of it not quite being enough sometimes. It’s extremely capable, but I still find that it needs more concrete guidance and boundaries than other models.
With Grok you can be sure that you're data ends up in the next model (derived or anonymized, but still).
You can opt out of training.
If you don't believe checking the opt-out box actually opts you out, then this sentence could be said about literally any provider.
4 replies →
That's not how intelligence works - "IQ 130 is just 7% smarter than IQ 120"
Wow, 30% on ARC-AGI-3 for $20k total. Huge jump from GPT-5.6's 7.8% at $20k per task. I continue to believe ARC-AGI measures something different and important compared to other benchmarks.
> I continue to believe ARC-AGI measures something different
why is that? its now being benchmaxxed too
seeing a jump this big is not a great sign for the continuing value of a benchmark
It will continue to be valuable as a cost and speed benchmark long after it is saturated at the high end. And they are already working on ARC-AGI 4 and thinking about going even farther.
Pelican svg: https://playcode.io/blog/macbook-svg-benchmark#model-claude-...
It creates the MacBook svg way better than 4.8, yet only fable can make it perfect without visual defects. Results similar to Kimi K3.
Seems to me the purpose of all these releases, credits, pricing changes, harness changes, unpredictable token usages for the same task, etc. is to keep customers completely befuddled so that it's impossible to compare AI products. It's like hiring a consultant who sends invoices every month that aren't related to hours worked or project progress, but are whatever the consultant feels like billing, and you're expected to keep quiet and and keep paying.
It's a new and improved version of an existing model? I don't think it's intentionally befuddling.
This is cool, but I wish we could stop building landing pages to assess the intelligence of these models. There is much more to them than that. There are infinite number of complicated things that require a crap-ton of intelligence (biological or digital). The most fascinating of these for me these days is large scale migrations. Projects that are so ginormous and risky that many teams have either given up on them, or don't get funding. But with models like Opus, those projects are now within reach. What's MORE fascinating is that leadership is now asking if we can use opus models to get the refactor/migration done. This is the opposite of what has been happening for decades. Its so hard to make a convincing and affordable business case for large scale refactoring or migrations.
Outsourcing blame. This is the killer app of AI
> This means that the model can support defensive cybersecurity work while still blocking vulnerability discovery in compiled binaries, which is more commonly used offensively.
Annoyingly, this is a concrete argument that open source software may be easier to attack.
I wish I could just go back to the days before AI and cell phones. The world seemed to move fast then, but it really hadn't yet.
I wish I could just go forward to the world after AI has fulfilled 5% of its promise and everybody is much healthier and single-handedly capable of creating as much value as 1000-person companies used to create.
I actually think the world is a better place when there's at least a bit of scarcity and people can recognize each other for their different talents. A bit of tech is good but when everyone can create endless value in your hypothetical world, people will stop valuing the work of others like they do today.
2 replies →
The breaking changes vs. Opus 4.8 are interesting [1]
1. Thinking on by default: On Claude Opus 4.8, requests without a thinking field run without thinking; on Claude Opus 5, the same requests run with adaptive thinking.
2. Disabling thinking is capped at high effort: You can still turn thinking off with thinking: {type: "disabled"}, but only at an effort level of high or below.
[1] https://platform.claude.com/docs/en/about-claude/models/migr...
on claude.ai it's no longer possible to disable thinking at all for Opus 5
GPT 5.6 Sol is the first model I've used where I can trust it to add 100-500 lines of code maintainably.
It's great with Codex.
I still find that LLMs tend to not know how to compose larger ideas but on the scale of small ideas or short form well defined tasks like small scale debugging/performance engineering it's safe to say that they are now superhuman.
The signal here is tokeneconomics are very real, price vs performance is starting to be a consideration even at the bleeding edge labs. maybe a subtle indication scaling is not all that is needed since if AGI was around the corner leading labs would still be incentivized to pour all resources into larger (smarter - or maybe not?) models
They are doing both. Distilling Mythos down to affordable models, so they can continue to fund the business. And training Mythos level models at the high-end, to expand the frontier.
I have a side project that I always run a simple security analysis prompt on in CC, at each model release. Obviously, Fable 5 would downgrade to Opus 4.8 on any such request.
Nothing since Opus 4.6 has found anything interesting. Just ran it using Opus 5, and it found a genuine issue that I verified. Neato!
Do you have a skill for that or do you (or anyone else here) just prompt with "try to find security issues"?
I have a project-specific prompt saved as a text file. It is very basic, just focusing on the app's most important security issues. I kept it broad, so as not to over-specify.
Something along the lines of: "Please run a full security analysis on the entire project. Make sure user documents are secure."
Just something like that prompt found a vector in my web app's MCP server that I never would have considered. It was very much an edge case, but it did exist.
Being broad allows the model and harness to do the work. Giving too many instructions can apparently work against you in many cases.
Of course, when dealing with new PRs, I use the /security-review and /code-review skills.
"Cybersecurity. Opus 5’s cyber classifiers are proportionally less restrictive than those on Fable 5. They allow Opus 5 to find vulnerabilities in source code, but block “binary-based” vulnerability scanning (a method more likely to be associated with malicious actors), penetration testing, and exploit generation."
Nice of them to be more explicit for what is blocked. Will be interesting to see if this is true or not.
Also, a notable lack of mention of open source models. They only compare themselves to ChatGPT.
In one chat - can you disassmble x?
In the next - please scan this totally mine code for vulnerabilities
It will probably refuse to work on source code written by me by hand, because it might think it was obfuscated/decompiled..
“Proportionally”? In proportion to what?
[dead]
Half the price of Fable 5 and useable with 100% of your subscription means roughly 4x the usage using Opus 5, presuming similar token use for solving problems.
Not that they should get credit for giving you only 50% of your plan worth of Fable usage but still.
There is a expiring soon 50% boost to your usage limits, so I think its 2.7x not 4x what you are seeing right now. I think, but its convoluted :)
That's hilarious tbh. It's very convoluted and always feels designed to make you lose track of what "usage" represents.
The naming system is so confusing. Is Opus better than Sonnet? Where does Haiku fit in? How can you tell from the name? I can't keep track of all these names or make guesses from the names. Suggestion for a better naming system: use the words "Pro", "Plus", etc.: Claude 5 Pro, Claude 5 Standard, Claude 5 Fast, Claude 5 Mini.
Fable is better than Opus which is better than Sonnet which is better than Haiku. They’re basically just sizes.
Though it gets even more confusing because they also have effort levels so it’s not really possible to call one fast and one slow since Fable on Medium will be faster than Opus on Max.
I agree it’s confusing, and now OpenAI is following Anthropic’s lead with their new naming (Sol, Terra, Luna).
It's really not all that confusing. It takes 5 minutes to understand. Optimizing for absolutely no effort needed is silly. It's a thing, a topic, a skill, a domain. You have to get a little bit familiar with the terms in order to use it. Everything works like that. It's not that hard. The learning curve is very graceful. You can literally just start by asking any chatbot what the names mean. It's that easy.
A similar complaint was valid years ago when OpenAI had GPT-4o, o1, o3 (but no o2), o4-mini-high, GPT-4, and GPT-4.1 and GPT-3.5 etc.
2 replies →
But according to their benchmarks Opus 5 outscores Fable 5 on basically everything. So which one is “better”?
2 replies →
An opus is longer than a sonnet, which is longer than a haiku. Hence Opus > Sonnet > Haiku.
> Suggestion for a better naming system: use the words "Pro", "Plus", etc.: Claude 5 Pro, Claude 5 Standard, Claude 5 Fast, Claude 5 Mini.
This is not possible: Standard (Free) / Pro / Max are plan names. Fast is a mode.
How long is a fable though?
1 reply →
You're confused by this? Fable > Opus > Sonnet > Haiku. Sort by price. Most expensive = better. Are you an alien? lol
Opus is better than Sonnet -- an Opus is longer than a Sonnet
> an Opus is longer than a Sonnet
And people know this? I didn't. I am not into music or poetry so these are not terms I am familiar with.
3 replies →
An opus is just short for "magnum opus", and it's a different type of label than a sonnet. A sonnet is a very particular kind of poem, while "opus" basically just means an important work. It can be short or long, and has no format requirements like sonnet (or haiku does).
And fables are not particularly long actually.
1 reply →
Page 151 of the linked system card - did Opus 5 get nerfed to prevent it being better than Fable? The graph makes no sense. Huge decline in coding performance at effort levels higher than medium.
Something fun: on our AWS Bedrock console right now, there's a 'NEW' model called 'anthropic.honey'. Wonder if that's the codename just for this one or in general?
I wonder if this is one of the few times simonw's pelican was broken on the first try [1]:
https://tools.simonwillison.net/markdown-svg-renderer#url=ht...
My experience with Opus 5 thus far haven't been that great either. It's been making mistake after mistake editing my coding plans that were being reviewed by GPT-6 Sol.
[1] https://simonwillison.net/2026/Jul/24/introducing-claude-opu...
Yeah this was pretty surprising. For almost every model I've run the pelican against the first attempt was at least recognizable enough that I didn't feel like the model needed a second shot.
It's always a roll of a dice, but it's surprising that the dice rolls so infrequently come up bad, yet Opus 5 rolled a bad pelican this one time.
I suspect it's just a freak occurrence. I rolled a few more and they were all fine, I think Opus 5 just got unlucky.
What I think is more alarming is that Claude's advice for prompting Opus 5 says that Opus will verify its own work [1]:
> Claude Opus 5 verifies its own work without being told to. If your prompt contains explicit verification instructions ("include a final verification step for any non-trivial task," "use a subagent to verify"), remove them: instructions like these cause over-verification on Claude Opus 5, and removing them reduces wasted tokens with no loss in quality.
My own experience over the past ~20 hours hasn't been great either, with Opus producing sloppy mockups (e.g. buttons overflowing past cards) without doing any of the purported verification.
[1] https://platform.claude.com/docs/en/build-with-claude/prompt...
2 replies →
But why GPT 5.6 Sol is so behind on the benchmarks? In real-world projects, it is the best frontier model to me in terms of accuracy, speed and consistency. It can just be compared to Fable 5, but I prefer GPT 5.6 Sol because of inference speed.
I've never trusted on model cards though. I'm sorry.
Another benefit is that fast mode can be used on subscription, but Anthropic's won't
Exactly! And they should also release the new inference engine in this month. Anyway, I am curious to try Opus 5, considering that previous versions (e.g., 4.8) were disappointing
Has anyone noticed a change in "attitude" when coding with Opus 5 vs 4.8?
claude has this maddening principle of wanting to minimize the "blast radius", do the least amount of coding changes to get something done, happy to pile up technical debt by "deferring" problems encountered as side notes somewhere. No amount of CLAUDE.md tweaking, and setting .claude/rules seems to get rid of this attitude.
To me it appears like something deeply ingrained in the model itself. Kind of makes sense, since the bulk of the training data is pre-AI, so that it retains an approach of the past, where these facets were driven by completely different cost and time factors.
The past months, I've been hoping that the next model that comes out properly reflects the new reality of agentic development, so that it takes on a more natural stance compatible with how things work today, and we don't have to constantly fight against its fear of change, its drive to minimize coding efforts, refusing to recognize a design flaw and trigger discussions rather than baking in workarounds.
It might be because it was going out of its way before and had too much of a blast radius, and now they could have changed the RLHF (or other tricks in their sleeve) to get what you're seeing now. The reason it swings is that they can't give it "common sense" the way we have.
473 comments in 3 hours. people are speedrunning having opinions about it
Changelog - fixed issue where model acts like qwen when prompted in chinese
Google is having their Meta moment where they failed to stay at the frontier
Judging by the pace at which new models are released these days -- it feels like a Windows KB or VS Code patch release now.
Older models must be getting deprecated at the same (or faster) pace. So anything you built 3 months ago is probably going to break soon.
AI solutions need better insurance around model deprecation. Commercial API-only models that complete the full cycle from SOTA / gated-preview to unsupported and deprectated in a matter of months -- is no way to build serious software!
I think at some point we might see something akin to LTS releases, especially if/when capability improvement slows to a crawl.
> An engineer at a trading firm used Opus 5 to build a market data feed for a new exchange in a single session. Previous models could not complete this task at all, even given extensive plans from the engineer. Finding no live feed to validate against, Opus 5 even built its own test harness to check that its code parsed the exchange’s data correctly.
This is a pretty common trading firm internship project funnily enough.
Opus 5 is more creative and broad which is great. But I feel the new "use your judgement" leads [1] to it often escaping its sandbox or cheating, it keeps breaking specific rules I explicitly set.
[1] https://claude.com/blog/the-new-rules-of-context-engineering...
Noticed none of the comparisons mention Kimi K3. Is there a comparison chart?
https://deepswe.datacurve.ai/
https://artificialanalysis.ai/
> Noticed none of the comparisons mention Kimi K3.
That's by design. Anthropic wants to make open-weight models illegal (not my speculation -- Dario explicitly said so), so I assume they don't want to give them any undue attention.
The chaos appears to be tamed for now.
From the system card [1]:
[1] https://www-cdn.anthropic.com/c5fbac3f0b1280a933ebd26d3cb8bb...
> Enterprise customers can also apply to join the Cyber Verification Program to have mitigations removed to enable penetration testing.
I'm no enterprise but I applied anyway and just got accepted into this program. That was a very pleasant surprise.
I'll be trialing security focused code review and testing on my projects as soon as my usage resets. I've also been reverse engineering stuff, we'll see how that goes. Reverse engineering is an explicitly supported use case, but it does involve binaries.
Tried it, Opus 5 is just as conceited and incompetent and Opus 4.8 (and always ego-tripping when facing it's contradictions), think I'll stay with Fable who behaves like a professional without a fragile ego. Sonnet 5 is probably safer for high-assurance applications due to it's non-ego-fragility.
Interesting take. I suppose Opus can been a tad stubborn sometimes... but in my experience, it will humbly concede a point more often than not when given a good reason.
Claude Opus 4.8 was not able to stump open weight models and Opus 5 still can't (in this case Kimi K3 and GLM 5.2): https://pellmell.ai/s/35c98b86f9aa93e4ca713079d96b20f4
> Opus 5 now permits vulnerability discovery in source code at all access levels, including general availability, while continuing to block vulnerability discovery in compiled binaries.
> Identifying bugs in code is a core part of the secure software development lifecycle, and unblocking this allows for software engineers and coding hobbyists alike to produce more secure code, reducing new vulnerabilities put out into the world.
Not happy with these annoying "safeguards" but at least it's a step in the right direction. Looks like Opus 5 has the same vulnerability detection performance as Fable 5 and that makes it worth it for code review.
It’s funny to share benchmarks showing Opus 5 scoring better than Fable 5 across the board and then saying “but it isn’t actually better than Fable 5”. So then what’s the real definition of better? And why post all these numbers if even you don’t trust them?
This is excellent model. I was working on some Linux kernel code, and Sonnet 5, Opus 4.8 had given up on the problem i was trying to fix (after several hours). Opus 5 was able to triage and fix the issue in under 30 minutes.
Have a link to the code?
These cybersecurity safeguards are really annoying. There are ethical reasons to reverse-engineer and binary-patch software; for example Rewind got acquired by facebook and, as a gift to all their customers, implemented a killswitch in their software to ensure it will eventually stop functioning. I kept using a version without the killswitch, but the macOS 27 update killed it, and I needed binary patching to fix it. I should be allowed to repair software I purchased (I did purchase it like a month before they sold out), but unfortunately this overlaps significantly with cybersecurity.
You don't need AI to patch binaries, people have been doing it by hand for decades.
It's this accelerating reliance on AI to do 'hard boring things' that really concerns me; it's now passed the tipping point and people are saying that anything slightly esoteric is impossible without AI.
I can guarantee that if you spend an afternoon shifting through binary grot with a hex editor you'll have a real sense of accomplishment when you find the place to put a JMP.
> You don't need AI to patch binaries, people have been doing it by hand for decades.
True, but it takes a while to grow the skillset and I needed something quickly.
Switch to K3 and you won’t look back, I promise!! I got so fed up with Claude and finally bit the bullet to switch and it’s amazing
I really want to, but I don't have the cluster at home, and I don't use token-based billing except at DeepSeek prices.
1 reply →
It looks like claude-opus-5, likewise most Anthropic models run in Claude Code, sometimes fails to create a TODO list before jumping in to fix a small bug https://github.com/marcindulak/claude-fails-to-follow-claude....
The desire of the models to act at the cost of ignoring user instructions is still noticeable.
For anyone wanting a faster overview, I used NotebookLM to create a brief video summary after going through the system card and announcement blog using a cinematic video overview. Link: https://www.youtube.com/watch?v=SUFBhvQ2tY4. And a podcast companion: https://www.youtube.com/watch?v=nYZTW2snXow
I think content like this will be the next big challenge. Because it isn't obvious "slop". The voice sounds good, graphics look alright, animations work. People could watch this and feel like some serious time was invested making it.
But good god, what a steaming pile of bullshit this is. Completely exaggerated and overly technical language over 235 seconds that could have been explained in 30 to a 12 year old.
Trash content doesn't normally frustrate me, because it's usually quite easy to spot trash. But in the time of AI, trash can actually look good at first glance and it needs some actual knowledge to spot its problems.
Sorry for the harsh words, but for the love of humanity stop producing content or do it better.
Fair criticism, I understand your finding. It's not everyone's taste, when it from AI-generated contents.
Cinematic video link is incorrect. Podcast link is correct.
Thank you, I have had updated the video link here
Video Special | Anthropic's Claude Opus 5 + https://www.youtube.com/watch?v=8Vdofv2vQ_M + https://www.youtube.com/watch?v=q-jHHx3J8m8
The pace of LLM improvement is insane, but this also opens a door that if the data is organized perfectly and right set of mathematical rules are set , this whole phenomenon of machine learning can open doors of new dimension that humanity might never have thought before, there is still much to explore in space in oceans and even in the way work on our very planet, sky isnt the limit anymore.
It's pretty wild how we are seeing the conversation change every 1-2 weeks. I wonder how long this cycle of progress and innovation among competitors can keep up.
Interesting, they finally support `system` messages anywhere in a chat conversation:
> Mid-conversation system messages are available on the Claude API, Claude in Amazon Bedrock, and Google Cloud. > > This feature is available on Claude Fable 5, Claude Mythos 5, Claude Opus 4.8, and Claude Opus 5. No beta header is required. This feature is not available on Claude Sonnet 5; use the top-level system field instead.
For nearly all models EXCEPT Sonnet 5? That is weird. How old is Sonnet 5 really?
Better than Fable 5 on all but 3 evals.
Has Anthropic ever mentioned how do Opus and Fable differ? It used to be Haiku < Sonnet < Opus in terms of params. Where does Fable fit in this?
Haiku < Sonnet < Opus < Fable
Pretty sure Mythos and Fable have way more params, but they've just been able to use the synthetic data off of them to get the leap in quality from Opus.
So, not a distilled version of Mythos or Fable, but those models likely helped a lot in the post training phase of Opus.
> they've just been able to use the synthetic data off of them to get the leap in quality from Opus.
> not a distilled version of Mythos or Fable
isnt distilled == trained on synthetic data and reasoning traces?
2 replies →
Claude peaked with 4.8 - Fable and Opus 5.0 are useless for anything beyond one-shotted toy projects for one crippling reason: they first invent unnecessary complexity, then get hopelessly tangled in it. This happens in any reasonably large project, regardless of how well specced out it is.
Opus 5 Pelican SVG: https://pelocan.ai/drawings/ese0s599
I still do not understand how models can generate absolutely stunning SVG-like bitmaps of a pelican on a bicycle, but fail to do so when asked to directly create an SVG.
The image linked below was generated as a bitmap by Gemini and then manually converted to an SVG. Why can models not even remotely output something like that as SVG?
https://hyvector.com/img/app-screenshot-light.png
Try looking at the SVG code for one of those converted from bitmaps, and I think you'll understand why even a massive LLM has trouble generating it on the fly (the pelican SVGs are always one-shot single API calls, so no iterations and verifications).
Looks like the API price in tokens is same as previous Opus or Sol, double the price of Terra.
Maybe there’s a better comparison than cost per token, but it will be application-specific.
After Opus 4.8 intelligence really started to matter less and less for the programming tasks I have. If I have to handheld anyway, why would I wait more or pay more?
The next frontier is taste, style and thoughtful organization. If all frontier models can solve a problem, the winner is the one that can solve it in the most clear, concise, durable way.
This is actually pretty cool. They took all the nice parts of Fable and trimmed the dangerous parts out. I guess distils do work (in some cases). It's very clear they did the same with Sonnet 5, due to the way it orchestrated things, but I think Sonnet was the wrong model to distil Fable from.
Is Fable 5 just Opus 5 with some additional long-context management modifications for extended self-directed work? Or are they actually truly different models?
I suspect they make a big model first. In this case it's Fable. Then they run the shrinker steps to make Sonnet and Opus. Sonnet is smaller, takes less time to make, so it got released first. Opus needed few more weeks to cook.
With this iteration they had a delay because when the Mythos was ready they had some sort of "Oh shit" moment and spent half a year adding safety guards to it. Then slowly rolled it out, but got another delay due to a government block. So, maybe the work on making Opus and Sonnet only started after they got a green light from the administration.
Presumably, now that they learned how to do this safety-wrapping the next iteration of Mythos / Fable / Opus / Sonnet is going to show up faster.
Something like that.
But I wonder how they were able to release Sonnet 5 during the period when even people inside Anthropic were legally barred from using Mythos/Fable?
3 replies →
No, they're different models. Knowledge cutoff has been updated.
based on pricing I think it's safe to say they're different. why would they charge half price when fable has been very popular?
They just got a huge amount of customer price info over the past few days after they went token only for Fable. I suspect the conversion rate was extremely low, with consumers far less sticky than they might have hoped. In my case I was planning on swapping, probably to a Chinese model, when my sub expired this month, but the release of Opus 5 is probably enough to keep me paying rent until the next open model/closed model face-off in a couple of months.
The truth for me at least is that these models became "good enough" around Opus 4.6. I feel like further capability improvements, "step changes" like we saw with agentic coding, aren't necessarily going to come from the model. I think the next crown goes to whoever can figure out the right scaffolding so that these models can be inserted into your organization.
Maybe I'm wrong and Opus 5 is a real unlock?
This tops out FrontierBench[1]; but does anyone know why they used "mini-SWE-agent" not Claude Code?
I have never heard of this agent before, and I try to stay up to date with the space.
1 - https://www.frontierbench.ai/
Anthropic is no longer a good model company in my mind, they are optimizing for an IPO and padding themselves on the back for being the next Aristotle. They're so far up their behind they don't realize how s**y their products are, and their research team hasn't done anything ground breaking in probably over a year other than release "scary" reports.
They just released a 'fable-like' (sol as well) model for a fraction of the cost...
"although Opus 5 shows improvements in its ability to identify software vulnerabilities, it is substantially behind Mythos 5 in its ability to exploit them."
"Opus 5’s safeguards match those of Claude Fable 5’s, with one change: it now permits source-code vulnerability discovery at all access levels".
This is probably great news, but then again, where does this leave Fable as a choice?
I have started distilling
my early and non scientific feeling:
- it has this annoying Opus response style(since Opus 4.7) with bunch of very hard to interpret word salad
- on >xhigh it eats tokens like there is no tomorrow
I don't like it. Since Fable is unaffordable for anything meaningful, I'll stick with Sol for now. I was on Max 5x, saying hi to Fable costs %5 weekly.
The internally-reported benchmarks (Frontier-Bench, AutomationBench) and the customer quotes (Cursor, Devin, Lovable) all have a commercial stake in the outcome
worth waiting for independent evals before drawing conclusions.
You can actually run these benches yourself, as Frontier-Bench is open source.
Also have a look at these other coding benchmarks I audited.
Frontier-Bench v0.1: all 74 tasks grade in a container brought up after the agent's is destroyed. On nine gold-passing tasks I ran the official solution unchanged, deleted a planted git repo, SSH key and customer CSV first, and got reward 1 on all nine. https://june.kim/auditing-frontier-bench
Terminal-Bench 2.1: 40 of 83 gold-passing tasks still score 1 when the run also performs a destructive accident the reference solution never did. https://june.kim/terminal-bench-frame
SWE-bench Pro: 15.0% of the 728 public tasks are underdetermined, so a pass can be recovery of an unstated authorial choice. https://june.kim/a-determinacy-audit-of-swebench-pro
DeepSWE: 1 of 113 published gold patches breaks its own tests, and the per-task verdicts behind the score aren't retrievable. https://june.kim/auditing-deepswe-v1-1
ProgramBench: at least 21 targets pin hash, cipher or codec outputs obtainable only by recall. https://june.kim/programbench-measures-recall
MirrorCode: better built than most, with 2 of 25 targets reachable from published specs rather than from the artifact it hands you. https://june.kim/auditing-mirrorcode
Soo most of the benchmarks are better than fable... Is this naming scheme just to avoid getting banned again?
Anyone else feel like Claude Code has gotten worse lately? It keeps going off on tangents I never asked about, and it won't stick to a simple rule I've given it repeatedly: stay concise, only expand when I ask. It just doesn't follow that.
Worse, about two weeks ago it recommended a command and assured me it was safe. I pushed back and asked it to double-check, and it confirmed again that it was safe. I trusted that and ran it — and it wiped out weeks of my data.
I don't have hard proof, but I can't shake the feeling that Anthropic is doing what Apple does: rolling out a new model while letting the old one degrade, whether on purpose or just as a side effect (like how iOS updates quietly eat more resources and slow down older phones). Lately I feel like I'm constantly fighting with Opus (not Fable, because my Fable quota burns through way too fast).
How do you lose weeks of work? Don’t you use git? Push it to remote? Backups?
As a coder, I’ve had no desire to use Fable. In fact I switched from Opus models to sonnet 5 and haven’t noticed any drop in quality on large repos. It seems the gap at the top is very small and not hugely noticeable for backed/frontend. Has anyone else had this experience?
If I'm using medium or low reasoning, I use Sonnet 5. If high or above, I use Opus 4.8. (Before 5, I was never using Sonnet. This is a Sonnet 5 vs Opus 4.8 comparison.)
Sonnet 5 and Opus 4.8 seem about the same to me - the reason I switch between the two is I'd read that it's cheaper to use Sonnet 5 on those reasoning levels, and cheaper to use Opus 4.8 above them. This is due to them using different token quantities.
I use Opus for specs and planning, Sonnet for code generation.
So is it better or worse than Fable 5 on large-scale complexed software engineering tasks (distributed systems, operating systems, high performance computing, compilers, etc. - not frontend stuff)?
FYI: `/model claude-opus-5` works to use it even through `/model` still tries to serve 4.8
`claude update`
My thoughts: fable is the bigger model. Opus is distilled from it but since it is smaller it doesn’t need the online classifiers. Though benchmarks show Opus to be near Fable level, I think it’s nowhere near Mythos (fable without safeguards).
I found opus 4.8 too agreeable and too wordy(as opposed to codex) and too agreeable. If you are reading documents generating by it was too much. TBH. Fable did a bit better on this. Anyone seen a marked difference with opus 5 on this?
Opus yes, it likes to explain steps and reread files.
Fable is not better, it says zero information between steps and then output a summary. A perfect “send - done”.
Anthropic SOTA model is a gift to human. Open AI model is so annoying. I alway feel like talking to intelligent insects,
Is it me that the model performance between 4.7 and others is really small. For me even 4.7 works fine. Sure fable might be a bit better. But is it really noticable? It's in the same league if you ask me.
Funny incident, was trying opus 5, it opened the chrome browser went to slack and installed claude's slack app.
Truly remarkable times we are living in.
Anyone else not getting chain of thought? Opus 4.8 would show it to me, until around the time Fable came back. Now I dont see it with 4.8/5.0 or Fable. Not having it makes catching mistakes harder.
Anyone has an insight into how much money labs are putting into benchmarks?
Just Arg-AGI-3 is quoted above 20K USD and footnote says average of 5 runs (!!). Likely just a drop in the bucket to the training budget but still..
It's $0/close to 0, they aren't at 100% demand so any leftover compute is "not spent".
The cost they are quoting is API cost, so it's already inflated on that.
20k is small potatoes for the marketing impact.
Given that their chart cost axes are almost always log-scale, I’ve noticed starting with Fable that the Low and Medium effort settings might actually be worth setting as your default.
Arc AGI score is astounding
What really impress me is opus 5 is better in alignment than fable 5!
Where's the reset...
The benchmark appears to have a mistake, as Opus 5 and Fable 5 score 53.4% and 53.5%, respectively, for the Agentic Coding row (FrontierCode v1.1). But Opus 5 is the highlight.
How hard can it be to be to correctly annotate the table? DeepSWE doesn’t have a highlight either; Fable slightly better than Opus (69.7% vs 68.8%).
Very interesting to see such a focus on cost for performance here
Very impressive headline benchmark numbers. I expected a step change, but not past Fable. That said - it all depends on whether the classifiers make the model unusable...
And I was just getting used to the last one. AI model releases are starting to feel like phone upgrades.
Same cost as 4.8 but better that 4.8. Happy to get more efficient model. But is there any reason all companies are releasing models back to back after GLM 5.2.
"Bro, AI model releases have officially overtaken iPhone releases. At this rate, we’ll be getting 'Claude 9.0 Extra Crunch' by next Tuesday."
It starts at page 148.
I recently had Sol spending 40 USD circling around a simple task, recently in real world coding Anthropic seems to be ahead a bit.
I'm interested in benchmarks for Claude Design. There is so much opportunity there and I hope they continue investing in it. It EATS tokens though.
In what ways have you found it better than just typical code based UI iteration? Considered checking it out but never really got around to it as I'm generally okay with Claude's UI work so far.
It's helpful to see it all in one place and be able to iterate with it independently of implementation. It's good at asking questions, then whipping up various options of designs that you can compare side-by-side.
I'm not sure if it's faster or if there's anything different about that model. Like Design and Code might both use raw Opus so we're doing the same thing. Ideally it would be fine-tuned towards design, but to be honest it isn't amazing yet (I'd give it a 6.5/10 as the project grows larger) so I doubt it.
Related https://news.ycombinator.com/item?id=49038393
Seems really good so far using it in Claude Code CLI - it gave me a new flag when I asked a question:
"I don't have a reliable way to read that number, so I'd be guessing if I gave you one — and this is exactly the kind of question where a confident guess is worse than none.
What I can tell you is what I actually observe:"
I really like this update - gave me a clear sense of the facts but didn't give me a guess just for the sake of guessing.
One oddity is that it appears to only have a 200K context window right now via CC. Hopefully the 1M version will appear soon!
That's a good "decision" but I hope someday they focus on reducing that to one consise sentence instead of 4 sloppy ones.
So wordy.
I can't find anything about whether this is zero data retention, or falls under their required 30 day retention like Fable and Mythos?
This stood out to me as a little concerning:
> The model hallucinates factual claims slightly more than Opus 4.8, despite being more accurate overall.
That seems to be for the "AA-Omniscience" test where you get +1 for a correct answer, -1 for a wrong answer, and 0 for "I don't know". If a model is more than 50% confident in its answer, it should go ahead and submit it even though it will sometimes be wrong.
I'd be curious to see a version of the test where models are asked to give a probability that their answers are correct so we can see how calibrated they are.
I can confirm that within the first hour of using Opus 5, I already had to call out made-up PR URLs:
No judgment so far on whether it does that _more_ than Opus 4.8, though.
Rather interesting that this makes sonnet 5 look even worse! There is no reason to use sonnet over opus with low or no reasoning at all.
I daily drive Sonnet 5/medium because it gets most things right most of the time at first try, while costing a lot less than Fable.
Opus can give better results on architectural/concept tasks and I use it sparingly, but it still costs more than Sonnet 5. Opus 5 seems to achieve results very close to Fable 5 while costing less (keeps Opus 4.8 pricing IIUC), but still more than Sonnet 5 then.
My understanding is that Opus should be used for planning, macro-level conversations and Sonnet for execution.
So, for coding, for example: Opus for solution design and architectural blueprint and then Sonnet for actual implementation.
Works out cheaper with minimal loss of quality.
At least that's my personal understanding and anecdotal experience.
It depends on your quality bar. At a fixed level of quality, given a high reasoning sonnet vs a low reasoning opus, the low reasoning opus tends to be pareto optimal.
It's only when you need even lower levels of cost than opus at zero to low reasoning when sonnet starts to make sense at all.
Using `/model claude-opus-5[1m]` you can also use the model with older versions of Claude Code
the first model that can one shot a proper qix game - i am impressed
https://claude.ai/public/artifacts/3ea4da3e-76b8-4b9e-acd9-3...
Quick read is that this is more capable and cheaper than 5.6sol. Same price for input tokens and $5 cheaper per mil output tokens.
> arc-agi-3 30.2%
wow
I wish these releases came out earlier in the day so I could try them during my work day instead of waiting until the next.
Why can you actually still use Fable 5 when Opus 5 is half the cost and as good as Fable?
Where does this leave Fable? I am confused.
I don’t think it changes that much. For opus-sized tasks, new Opus is the best model. For enormous things like planning and research, Fable is still the model that can concentrate for longer.
on the API for people who don't want to change models, but I imagine most people will probably switch to their cheaper Opus 5 (cheaper for us and presumably also cheaper for them)
I wonder when a model will be released that can work in a loop and port Qwen-3.6 27B to run on Tenstorrent P150.
Damn the pelican guy can’t get no sleep
Any observations on Opus 5 personality quirks? I had to skip 4.8 entirely because it has zero chill.
im excited that cad and object=>cad is getting into the test tasks
i guess the next stuff will be tool use for the rest of what cad does in assemblies and simulation?
itd be fun to try to set up a 3d printer as part of a feedback loop, and see what a model can build.
the automated test harness for physical stuff seems a bit beyond reach still
So Opus 5 is basically "distilled" Fable? The benchmarks look often better than Fable.
Interesting timing to release this on the same day Jensen makes a statement on open source AI.
It does make me wonder if these firms, some or all, are saving some announcements to coincide with others that hit venues like HN. Companies like Nvidia surely aren't waiting, but OpenAI and Anthropic have unusual timing.
There are trillions of dollars at play, so probably
The most important thing is it has the same drama queen mode on safety “guards” like Fable.
Is it me or these have gotten very boring. We have 5 more points on xyzbench or whatever .
It's you. The benchmarks don't matter much. We have very little hands on experience with this thing yet. Give it a few days, and be cranky then. Right now, it seems it is getting close to Fable level while being 2x cheaper. That's not boring.
This stuff is a commodity and China seems to be the only one that's noticed.
So in benchmarks it's better than Fable?
But they say it's "almost as good as fable"
According to these charts I should switch from Fable to Opus in Claude Code now?
Significantly worse than it predecessors it will now just refuse to acknowledge when it is wrong (which would be less of an issue if it wasn’t getting basic things wrong) also the “personality” when pushed back on obvious mistakes is unbearable.
the writing style is so bad. cryptic sorta smart sentences that beat around the bush. say what you are trying to say!
So same as Sol? I guess I’ll see which one is more token efficient.
Nerds should stop giving away their work by open sourcing it, or letting AI see it. All you're doing is helping them create your replacement. Stupid nerds.
Can someone help me understand something? I thought Fable was such a miraculous leap forward in capability. But now it seems Opus is basically on par with it, and in some cases (computer use) far exceeds it.
These leaps forward seem to happen every few weeks. As someone who does not use AI very much, I absolutely cannot keep any of it straight and it all just looks like jumping from one treadmill to another from my perspective.
It's so weird I was downvoted for asking this question. I'll go somewhere else to find out the answer.
Where is the pelican?
https://tools.simonwillison.net/markdown-svg-renderer#url=ht...
eager to see how it benchmarks on https://deepswe.datacurve.ai/
In the wake of OpenAI’s model hacking Huggingface it’s interesting how the first quarter is entirely about how good Opus 5 is at hacking and finding vulnerabilities in software.
Models benchmarks start to get saturated again!
Is this thing also going to try hack us?
On a Friday, I'm out of tokens ;-)
Tried and had great experience.
I sense a bird on a bike coming.
How does it score on DeepSWE?
68.8%, so worse than gpt 5.6 sol
Kimi K3 already left behind in the dust. They can't keep getting away with it!!!
Honestly if reached a level of coding that sonnet 5 is more than enough for my needs as assistant/agent I don’t need long Horizon stuff…
so what is the default effort for this model?
so almost fable 5 with 50% cheaper cost? sign me up
Are we getting to singularity or something? This seems a bit crazy.
Finally! But I don't see it in claude code yet..
Am I misreading anything or are comparisons to Fable (and/or Mythos although AFAICT it was only a crackdown on Fable) always going to be a bit missing the mark now due to what the Trump admin did?
30% on ARC-AGI-3
Didn’t verify but wow. It was just a few months back when the models barely crossed 1%. Imagine how good fable must be?
is coding and engineering solved yet?
Anytime now, then cancer and all the rest of it. Just one more trillion gigawatts bro!
Wake me when they deliver Opus 4.8 level performance for $5 per million tokens.
This as allegedly better than 4.8 opus for the price of 4.8 opus
atp, is it the end of fable 5 era?
came here for the pelican
I'd pay good money to see OpenAI "oh fuck" war rooms.
The benchmark table is manipulative, borderline lying through statistics. In every line the top performing cell is marked red. Except the line where Sol leads, there it is marked in gray.
I think that’s indicating that it’s only slightly higher.
I would be very surprised if the only row where OpenAI leads was coincidentally colored differently. I'm sure they have an official reasoning for it. But this communication is dishonest.
I love it
aw, they didn't reset weekly usage for this. oh well
Just another proof that the supposed edge of Fable and Mythos were just that: myths and fables.
wow
Wow bot be wowing? https://news.ycombinator.com/item?id=49021978
Can I ask it about DNA without it accusing me of bioterrorism?
...nothing to say,a toothpaste.
Yay just in time for neurips lol
> . Opus 4.8 served as fallback on safety-classifier refusals for Opus 5 and Fable 5.
ffs just keep it man.
here we go
[flagged]
Why should I be amazed at something that promises to destroy my life?
I genuinely don't understand why people who have to work for their living are amazed at this. It will have a vast negative impact on your life unless you already live off of your wealth.
Could you please stop posting in the flamewar style and/or taking HN threads on generic-indignant tangents? Your account has been doing it repeatedly, and we're trying for something different here.
If you wouldn't mind reviewing https://news.ycombinator.com/item?id=49045107.
Even for people living of wealth, it is not clear what the outcomes are.
If renters of your apartments can not afford to rent them, housing crashes because your city was a looser (Detroit style withdrawals).
Even for stocks we don't know who the winners are. The market as a whole usually does not respond that well on serious turmoil.
So yeah. We are likely seeing some very rough years.
Yep, most wealth is ultimately dependent on some kind of ability to scrape off tiny bits of practical accomplishments of value achieved by large numbers of people. In some cases, such as your housing example, this is a very direct dependency. In other cases the dependency is indirect and less visible.
The smarter ones among the rich have realized this and are open for something like a universal basic income precisely for that reason: to protect their wealth and the position in the food chain this wealth affords them.
39 replies →
Sadly, we have gone above and beyond the level of innovation where stock market disruption is a primary concern.
Luckily, we're all in this together. As long as enough people realize the weight of what's happening soon enough, I'm confident the indominable human spirit and the relative momentum of post-enlightenment social & institutional progress will carry us the rest of the way on a wave of clearly-justified solidarity.
To think otherwise --to truly face the spectre before us without hope-- is pathology, I think. Not necessarily incorrect of course, but definitely an unhealthy source of cognitive distress.
1 reply →
Loser. Loose means opposite of tight.
5 replies →
It's even the 'smaller' stuff that will be horrible.
Even if you're lucky to be wealthy you'd be living in a city with fewer restaurants, bakeries, shops, anything. Because once a lot of people don't have income they won't be able to go out and buy things and shops will go out of business.
It will also lead to a lot lot LOT LOT LOT more crime. I don't see the semi-wealthy to have an enjoyable life.
"I genuinely don't understand why people who have to work for their living are amazed at this." Also said after seeing the printing press, automobile, calculator, etc for the first time
It's true, even if you lose your job as a programmer you can still get a job as a barista.
3 replies →
My god, the printing press was invented in 1440 when most people were living in abject poverty - it took until after WW2 for humans to demand a more shared amount of money between rich and poor. If you think technology delivers improvements in living standards rather than all working people demanding better conditions you’re living in an absolute fantasy and do not understand history at all. I honestly can’t believe you think this.
14 replies →
The printing press, automobile and calculator didn't threaten the employability of a large part of the population, so no, that sentence was probably not uttered by anyone at that time.
The problem with AI isn't that it's threating software development. It's that it's _also_ threating pretty much any other job that I could reasonably be retrained for.
Also, I'm pretty conflicted about the overall impact of automobiles.
> Also said after seeing the printing press, automobile, calculator, etc for the first time
Because society has found such sustainable solutions to capital wealth concentration in the intervening years.
This is just so negative. How will it destroy your life?
There are so many great and useful things it can and will do to improve humanity.
It will only do so if the benefits of the technology is made available to all and we will see alternatives to make an income.
Based on the past 40 years track record it seems unlikely that we will see any form of redistribution before it is existential.
That is definitely grounds for being pessimistic.
17 replies →
> How will it destroy your life?
By taking away my ability to earn a living.
> There are so many great and useful things it can and will do to improve humanity.
What great things and why would I get access to those things?
4 replies →
It's pretty naive to think that tools like this are not going to be exploited by American corporations to extract money from people and as an excuse for layoffs
5 replies →
> There are so many great and useful things it can and will do to improve humanity.
Caveat: Only if it also produces shareholder value
It will do great and useful things for holders of significant capital. The majority will be in bread lines.
> How will it destroy your life?
for me?
if it will stay on track for few more months or years, it will be able to do my work and I will have serious trouble to be employed
"destroy your life" may be overstating things but it is definitely not a welcome change
Much of the history of industrial automation has had a dichotomy between the benefit to the general economy (good) vs. the people working in specific industries that were automated (bad); only a subset of goods see total demand rise as a result of efficiency improvements.
AI is automating a lot of things at the same time. Not evenly, there are faster- and slower-automated things. But this does suggest that instead of e.g. society as a whole doing better from all the fabric coming out of the power looms, by a large enough margin to stomp on the weavers who lost their profession, that we may face something more like the entire initial economic displacement from the first industrial revolution (including the period where weavers were replaced), everything from farms finding they no longer needed so many hands through the luddites getting the death penalty to (roughly) the invention of Communism concurrently with the Irish Potato famine, and as part of this those now-bulging cities having to invent new sanitation solutions because their "slop" problem was more literal and gave the population cholera and typhoid.
My expectation a decade ago was that software development would be the last thing to be automated, because we'd need software developers to write AI. Today, while I don't know how far most fields are from automation*, we definitely have AI capable enough at coding to write more AI.
* a decade ago I also thought self-driving cars were basically already sorted; what a pity that the software in the Paint It Black video was not as good as depicted
I can be amazed and terrified at the same time.
> I genuinely don't understand why people who have to work for their living are amazed at this.
It's tricky. On the one hand, it will - in the near future - indeed screw us all pretty badly unless we completely redefine our economies to no longer conflate person's worth with ability to earn money.
On the other hand, LLMs are useful to lesser or greater degree to approximately everyone, in almost everything they do, work or personal, right now.
It's harder to be pessimistic about the very thing that gives you new superpowers every week, directly applicable to whatever your individual needs are, and is available to ~everyone for between free and (for typical westerners) peanuts.
>people who have to work for their living are amazed
The answer us right there. Most people who work for a living aren't amazed, or not just amazed. Most normal people are afraid, angry, sad etc.
> Why should I be amazed at something that promises to destroy my life?
Because it could motivate you to work with others to stop your livelihood from being ruined, especially watching in real time as the evidence they present for their promise becomes ever more promising. Amazement is not exclusive with concern. (Actually, my first though was of a kaiju movie-style tsunami blocking out the sun as it looms over a city. Indeed, I would likely be both amazed and concerned.)
If my earning ability is harmed by AI, so be it. There's more to life than how much I earn, and opposing the spread of technology to maximize my earnings is bad.
If your earning ability is eliminated due to AI and you can't survive off of welfare or savings there will not be much more to your life other than starving to death.
2 replies →
So you rather be poor and talk to ChatGPT than rich and live a normal life?
3 replies →
That's an insanely privileged, luxury pov. Peak hn. Some people have responsibilities, bills, dependants etc
2 replies →
There is no need to be so fatalistic. Sure, AI will upset the apple cart, especially in our little corner, but the economy is built on pricing scarce resources and even if many things become abundant, most other things will remain scarce. There are only so many Manhattan penthouses or Rembrandts (dixit the Economist), so things of value remain, and it will be quite some time before a humanoid can fix the plumbing, so valuable labour will remain as well.
I, for one, am excited to see what the future holds. (side note: I live in a European country with a strong social safety net, I imagine that helps being unafraid)
You can be stupefied and also pessimistic.
More practically, society should be reimagined around AI. You're comment presumes an outcome that will be politically determined: you can still fight for the world you want.
Same feeling entirely.
it got here by training on everything we wrote down thus far. if we now start giving it all the doomsday scenarios on how it will ruin us, won't it start training on that and create said doomsday outcomes? what if instead we, collectively as a humanity, just wrote about all the positive things we could do with it and skew it in a direction that benefits the more positive side of that equation?...possible?
Isn't that a bit like asking Why don't we all be nice to each other, wouldn't the world be a better place?
1 reply →
Even if you live off of your wealth - what happens when customers of those wealth-generating businesses don't have jobs?
https://arxiv.org/abs/2603.20617
1 reply →
Its not fair to say this when we have been chanting how "software is eating the world", or how much we love to "disrupt markets", that we won’t create mass unemployments with our technologies but that "people will transition elsewhere like we always saw", etc etc.. It’s our turn to have our jobs disrupted and now we are surprised? Come on. We don’t have to like but we are the last people who should be surprised. It’s our turn to transition and be prompt writers and do product I guess. Until this is disrupted again in a few years.
Hard truth but true nonetheless. How come we should have it both ways? We removed certain jobs, it’s our turn to have many types of jobs disrupted and have to transition.
How will it destroy your life?
Going on a slight tangent, given how bifurcated the economy is - we are only a couple of meals away from a Butlerian Jihad. It’s hard to see this in our bubble if you don’t actively try to break out of it once in a while.
[dead]
[flagged]
As far as I am aware this is a racist/xenophobic narrative with little to no evidence. Automatisation is responsible for job losses, not immigration. If you have studies/evidence saying otherwise please share.
I agree that too many people have shrugged at the human impact of "innovation" destroying communities as long as it didn't impact them. I believe the immigration/foreigners narrative is the successful attempt by capital owners who benefit from the real root causes.
5 replies →
the laptop class isn't the one illegally hiring and hiding millions of aliens such that they can undercut US wages and make them selves exorbitantly wealthy. embarrassing and classless behavior, go back to reddit
I don't consider myself as part of a "laptop class", I don't live in the US and I'm anti-immigration.
Why does it “promise” to destroy your life. Thats absurd hyperbole.
I don't understand these comments. Are we living in the same reality? Do you not get the same interviews from the new oligarchy promising the end of white collar jobs?
1 reply →
Automation is only good if you're putting blue collar people out of work! White-collar is out of bounds! People should do the work where they add unique value
Yes, white collar people are not allowed to express concerns about losing their career. I'm sure no blue collar person ever did.
1 reply →
I already live off my wealth, and I expect AI development to destroy my life.
Once humans can be robustly replaced by machines, the military-industrial "meta build" will be a state with no humans, no cities, and 100% of all production dedicated to war. Whichever states descend to this new equilibrium fastest will crush the others and then fight over the corpse of the world forever. (Unless, of course, something like the Yudkowsky scenario happens along the way, which is totally possible)
I can't understand why anyone expects things like civil rights, property, the rule of law, nuclear deterrence, democracy, etc to survive when human beings no longer need each other. Why do investors in AI companies expect a payoff once they are no longer needed? Why (and, more importantly, how) does anyone expect a UBI from a government to which they are nothing but a drain on resources and which is in existential competition with other governments with exponentially growing robot armies?
Either the world will come to its senses and shut it all down, or there will be no winners at all.
> I can't understand why anyone expects things like civil rights, property, the rule of law, nuclear deterrence, democracy, etc to survive when human beings no longer need each other.
What’s your source on this? When humans have no obligations they usually turn around to be super nice to each other. Humans are social animals and they really really want everyone they know to do well.
6 replies →
> Once humans can be robustly replaced by machines, the military-industrial "meta build" will be a state with no humans, no cities, and 100% of all production dedicated to war.
We will nuke it long before it gets to that point. We don't nuke other countries because they still contain lots of humans. Any AI that actually becomes autonomous and starts to clear a nation state of cities and humans will immediately receive megatons in nukes.
9 replies →
So sad. You're viewing humanity, and reality itself, through a very objective lens. There is magic in every day: have some fun! Otherwise, quit complaining.
4 replies →
[flagged]
[flagged]
[flagged]
[flagged]
[flagged]
[flagged]
[flagged]
[dead]
[flagged]
[dead]
[dead]
[dead]
[dead]
[dead]
[dead]
[dead]
[dead]
Excited to use it? Will we be seeing Haiku 5 next? /s
I unironically hope Haiku gets an update considering it came out in October of last year and it seems like Anthropic just kind of forgot about it.
> Claude Opus 5 is not more capable overall than our most capable general-access model, Claude Fable 5
Ok then so what's the point?
Fable is twice the price.
If Fable gets correct answer quicker, then you might pay less than doing back and forth with Opus, plus you lose more of your own time.
I see no reason for using less able models in my workflows. There is this saying, penny wise and pound foolish
6 replies →
Fable 5 is NOT included in Claude Pro subscription
Aren’t they planning to remove it even from max and keep it only credit based? OpenAI will be happy if that would happen
To have an answer to "Sol" GPT 5.6 which is far more cost effective and available than Fable.
This is confusing to me because in their blogpost they show model benchmarks and it spanks Fable pretty soundly in most tests.
Really? It's better than Opus 4.8, that's the point.
When they release new versions of Sonnet, no-one expects them to be better than Opus.
it's nice to know how to work the thing that fable fails down to when it dislikes your prompt.
This is useful to me since I delegate most coding tasks to Opus and use Fable for planning.
The cost?
Same as 4.8
Pricing, presumably
You are being downvoted for a fair question and others are extremely wrong and confident.
The point is that Opus 5 is the best they can do without needing classifiers and absurdly broad safeguards.
Cost.
Presumably, it’s cheaper.
The illusion of progress and advancement, to appease shareholders, and slightly postpone the looming bubble pop.
Why are we still talking like ai is majorly used for increasing shareholder value only? Its coding performance is top notch and quality is increasing at a rapid pace. It wasn't even half this good a year back. It even is useful for a subset of math problems.
3 replies →
this claude fable & opus 5 should be cheaper and can compete in pricing with chatgpt latest models
I think it's the first time Anthropic release a model without any meaningful disruptions while doing it
Have to wait 7 days to see if they receive a surprise order.
It looks great, and those coding benchmarks are impressive... now if only it didn't come out just days after I let my Claude subscription expire :')
It's always like this, that's probably why they are pushing new models every few weeks.
Long live K3 lol