Comment by user43928
19 hours ago
For comparison with hosted models, GPT 5.6 Luna scores 67% on DeepSWE, compared to 59% here for Qwen.
Luna is $0.20 / $1.20 vs $0.16 / $0.47 with Qwen.
19 hours ago
For comparison with hosted models, GPT 5.6 Luna scores 67% on DeepSWE, compared to 59% here for Qwen.
Luna is $0.20 / $1.20 vs $0.16 / $0.47 with Qwen.
This is a good counter argument. But you have to note that this is after OpenAI cut Luna costs by 80%. If you compare launch pricing, Qwen probably comes out ahead on a cost-performance basis.
The luna cost cuts were real though, not a one time promotion or something, due to some optimization (probably distillation?) that openai did.
You assume that openai's inference is profitable and that they aren't just trying to bolster revenue before their IPO.
The only indication that openai is profitable comes from openai (whom I wouldn't trust with any statement, especially when it comes to profitability).
In fact there is evidence that inference is not profitable simply because the rate of losses doesn't seem to reduce as revenue increases: if inference had great margins, we would expect that as revenues increase, the amount of spend on training reduces as a fraction of total expenses. Since the loss-making fixed costs shrink as a fraction compared to the profitable inference, we should expect profitability to rise with total revenue.
However, all leaks of openai's numbers seem to suggest the opposite: as revenues increase so do the losses.
11 replies →
what if it was because of quantization and they haven't released the new benchmarks for it?
Anything which changes the model needs new benchmarks I guess to compare with other models, otherwise you can benchmark Fable, and distill it to student model and keep claiming this is the Fable model
1 reply →
Was it?
Given the timing, I think they A. shat their pants since Deepseek flash just came out with insane pricing before the price hikes, and B. Anthropic is really struggling in model tiers below opus.
It was smart for them to cut prices regardless of whether they had 80% efficiency gains or not
Why would anyone car what the launch price is? Comparing launch pricing is just an odd thing to do.
Because labs can learn to optimize inference post launch, plus can move to use bigger/better clusters depending on demand. It is not impossible to imagine Qwen cuts prices further with QAT/MTP-like improvements.
>If you compare launch pricing
Why?
Those prices are just tokens? Since each model uses different amounts of tokens to do the same thing, it's a misleading price that often makes open-weights look more competitive than they are, since most open weights models use dramatically more tokens and time to complete tasks than many frontier models.
In Artifical Analysis's cost per task, Luna(max) costs $0.05 per task, and Qwen 3.8 27B costs $0.25 per task, a 5X increase. We'll see how 3.8-flash-next does.
the important thing is that Qwen 3.7 27B will run unlimited jobs on my consumer grade laptop at 60 tokens/second for free, forever, in about 1-2 years
Thats only important if running it locally is critical for privacy reasons or just as a hobby.
Time has a cost in business. If a model needs 30 million tokens to achieve a similar result as another that can do it in 10 million, that 60 tokens per second will take a long time.
Sounds like discrete propaganda