Comment by drnick1
2 hours ago
Why does an open weights model cost nearly the same as GPT5.6? $1.14 vs $1.23 on the cost index. Since you can't presumably run this on your own hardware given the model size and hence gain other things like privacy, I don't see any reason to move away from GPT at this rate.
It's not enough that it's better?
Many providers will host it and will compete on price. It also can't easily be taken away because one company (or one government) decides they don't want it around any more. People can fine-tune it for particular workloads.
They cherrypicked benchmarks. The ONE weighed benchmark where is beats Opus5 by 0.1 points is what was linked because that's how propaganda works. The Agentic Index that includes the full benchmark suite has it in 5th place.
Might as well use gpt-sol.
The whole industry cherry picks benchmarks.
I stopped paying attention to self-published benchmarks when Apple started using those non-sensical performance graphs with "relative performance" as a vertical axis when announcing a new chip.
> It's not enough that it's better?
It's barely better, and barely cheaper, not really enough to challenge the status quo IMO. Half the price for basically the same performance would be a much stronger value proposition.
What status quo? Just look at Openrouter's rankings: https://openrouter.ai/rankings
Things change radically month to month. Nobody is remotely close to capturing the market or having any kind of stability over time. People move around quite a lot, often to sidegrade within a generation. Just playing fly on the wall with discourse would be enough to tell you all of this, even without the data to back it up.
3 replies →
For one thing, providers of open models can't arbitrarily increase their prices without facing competition.
But given the extremely low cost of switching, why wouldn't you use the cheaper one if they're comparable?
As low as it is, switching between providers on OpenRouter is still lower.
That said, it's a fair point. For me, it boils down to things covered here: https://earendil.com/posts/session-portability/
Things like obscured reasoning traces.
Speed and reliability.
Why does an open weights model cost nearly the same as GPT5.6? $1.14 vs $1.23 on the cost index.
What cost the most in API. Input, Cached Input, or Output. There you have your answer.
Unfortunately, we have moved so much of the actual intelligence of models towards reasoning, what results in some models getting good scores, but this is because they are dumping a insane amount of reasoning tokens at the problem.
So a mid priced model, with heavy reasoning output, cost the same as a expensive model, with medium reasoning output.
Before the GPT Luna price drop of 80%, you actually had the same price if you used Luna High and Sol Low. With the difference that Sol Low was insane fast, and often way better code.
https://deepswe.datacurve.ai/
Do not look at the top score but more what is on the horizontal axis as you go down. Sol Medium is frankly, was the best performance for dollar, until that Luna price drop. I will even argue that despite the higher price, Sol Medium is still way better despite Luna Max being cheaper. Or Opus Low, one of the better values also.
What do you notice? Is that those models all have a high intelligence start point for their low setting. So that means they do not rely as much on output tokens aka thinking.
Qwen Max is their large model - over a trillion params. Similar to Kimi K3 in size. Qwen 3.8 27B is going to be more accessible to your own hardware. I'd say that Qwen Max is not approachable for the majority of people and companies to self-host.
Why should open weights correlate with cost? Cost correlates with the expense of running the model more than it does to the expense of developing the model.
Running a large model on rented GPU is still meaningfully more private than handing your chat logs over to FAGA
The acronym is new to me: Facebook, Anthropic, Google, openAi?
You said it yourself, model size and hardware. Big models cost more (good optimisation reduces things slightly, but they still need the hardware).
GPT5.6Sol completes the suite in 70M tokens, while Qwen3.8Max needs like 145M tokens. So this is a case where models like Qwen 3.8 and Kimi K3 use a lot more output (reasoning) tokens, go a good bit slower, so they can ultimately achieve a better intelligence score than if they went more quickly.
There are a couple of frontiers (ok bad word, maybe categories) in open weight models.
These Qwen 3.8 and Kimi K3 style models aren't trying to win on price, they're trying to compete on intelligence and capability.
Models like Deepseek V4 Flash (updated this week) are $0.03 a task, or 50X cheaper than Qwen3.8/Kimi K3, and 100X cheaper than Fable, while offering stunning intelligence. That's a different frontier for competition, and perhaps one more interesting for someone who wants to see them compete on cost.
it's a big honking trillion some parameters model. it's not cheap to run