Comment by mirekrusin
12 hours ago
Open-weight models
Kimi K3.1 Pro: −9
DeepSeek V4.1 Flash 1017: −13
GLM-5.3: −13
MiMo-V2.6-Pro: −16
Proprietary models
GPT-6 Astra: +4
GPT-6.1 Sol: +3
GPT-6.1 Terra: +3
Claude Opus 5.5: 0
Claude Sonnet 5.5: −2
Claude Fable 5.1: −3
Gemini 4 Argon (high): −3
Muse Wonder Full: −6
Claude Haiku 5.5: −7
Muse Wonder: −9
Gemini 3.8 Flash (high): −10
Grok 4.7 (high): −11
Is there affiliation with OpenAI or coincidence?
The eloquence of numbers is pitiless; thanks for summing them up. There are players filing for IPOs who don't want to lose appeal while contrasted to open-weight counterparts, which can be run on prems, or accessed as a service offered for a bargain.
More to the point, I found fable to be actually more "intelligent" than opus once you step completly outside of the technical side of things (and I don't only mean code here), so for me it's hard to judge.
Eg asking "here are all the contract I'm subscribed to, here is what happened, is there any insurance that covers me and if yes which one is the best to make the claim to and why, and provide all the article to prove why and how i'm not in the exclusion list". Try it with random stuff of your every day life. Opus is both great and not great.
Yeah, this smells. Ain't no way Terra went up and Deepseek so far low. I have had the later beat the former on multiple criteria by simply being less distracted.
There was a thread here about families of models responding in the same patterns to specific queries like "say a random word", and it takes less than two minutes to verify it is true. It'll take a bit of effort to not convince me this change wasn't reverse-benchmaxxed to screw everyone other than OpenAI. I'll be less distrustful after everyone is done IPO-ing.
My experience is not consistent with yours.