Comment by ndm000
10 hours ago
If the scores on Artificial Analysis really do mean that this model is better than GPT-6 Astra and Fable 5.1, that means that China is more like 6 weeks behind the US in frontier capabilities rather than 6 months. Which means that the US lead is all but gone.
> Which means that the US lead is all but gone.
You can see in this discussions the goalposts being actively moved to desperately create a notion that the lead is still there.
You even see people in this thread referring to imaginary wonder models that they expect US companies to have under development that are supposedly leaps and bounds more impressive, albeit the only reference there is to their existence is the commenters hopes and dreams.
I think people need to realize that there is no longer a cutting edge company that is expected to dominate, and among the multitude of AI companies in the field competition is so tight that no one would even be surprised if tomorrow some other company far away from the US was able to crank out the top performing model, or something like a small local model that outperforms mid/high tier frontier models. The local model alone would be a death sentence to US companies, who are still burning through 10x their revenue.
If true, it’s quite a bit more impressive than many people seem to think it is.
It’s incredible how little GPU compute they have to work with compared to the US frontier labs.
People thought the models are getting more intelligent via better architectures but all that really happened is that tool calling became a first class feature that every model is expected to support and RL training with coding harnesses became the standard.
Two years ago, tool calling was always considered an extra feature on top that some models where good at while other models completely ignored tool calling.
Meanwhile the discourse about how intelligent these models have become and how this is a major step or shift in model intelligence would make you believe that these models suddenly have a more powerful architecture when in practice model optimization has lead to faster but less expressive models that are still more intelligent, meaning that the model architectures didn't really experience meaningful shifts, it's that the training methods had a massive shift away from supervised training which inherently constrains the models.
The AI labs don't have a monopoly on the outcomes of RL training because RL is significantly less dependent on private data. Data helps massively for bootstrapping, but beyond post training, it's up to the model itself and every AI lab has a model so it is much harder to differentiate between your competitor if the RL training environments are the same (I specifically mean coding harnesses and Linux command line tools) because the environment here is the source of the "synthetic" data and everyone has equal access to it.
Ant/OpenAI both have unreleased models that are seemingly much more capable than what they offer today, so I'd say that's pretty unclear.
Why would they not release those models, since it would help them become profitable and beat the competition becoming the one company rule them all? Oh yeah, they are so dangerous. Well Anthropic revenue comes from only few big corporate customers, so just make them Golden Partners etc. and give them access to super-models. I doubt Google and Meta wants to crack web security.
Besides, if we start imagining super models US companies have in secret, why would we expect Chinese companies have them? Why would not Moonshot have their own super intelligence hidden in the cellar?
Vertical integration. They want to control all sectors, that's why OpenAI and Anthropic are slowly rolling out solutions for law, medicine, design and so on.
Are these models in the room with us?
Don't be snarky. This is very well known, including by public disclosures by the labs themselves, along with countless reliable testimonies by people who would know. It's actually a huge governance problem we're yet to solve.
Mathpocalypse was 2 weeks ago
I have an unreleased model that is more capable than those unreleased models (by about a gazillion times)
Perhaps you missed this part:
> Open-weight principles. These evaluations frame our open-weight release decision. First, we aim not to release open weights that would extend the absolute frontier of cyber capability. K3.1 Pro's offensive cyber capability is, by design, below what our pipeline can produce, and remains behind that of frontier proprietary models. Second, we intend to progressively raise the cyber capability available in open weights. We believe making K3.1 Pro available in a non-discriminatory way benefits people who need dual-use capabilities for defensive purposes. When a capable model is open and runs on infrastructure its users control, many more teams gain the means to audit the code they maintain — including legacy systems that rarely receive dedicated security reviews. We believe that letting this capability diffuse gradually — rather than concentrating it in a few proprietary systems — ultimately makes all software more secure.
I don't think Chinese labs are behind at all. I think the tell is that they keep releasing models 2 weeks after new American proprietary models that match or beat them. Suggesting they're sitting on much higher capabilities and releasing just enough to undercut
Appear weak when you are strong, and strong when you are weak. Sun Tzu
We could learn a lot from China. They know how to keep their own home grown billionaires on a leash. We in the West can't even do that.
Could be part of the reason that the frontier models are getting the administration to go a little harder against Chinese models.