Comment by minimaxir
12 hours ago
Hy4 apparently has ludicrous traction on OpenRouter already (https://openrouter.ai/tencent/hy4-preview), with trillions of tokens processed in a couple days: more than GLM 5.3 in a week. That said, it's relatively cheap with a 5% cache cost when everyone is still doing 10%/20% cache costs, so Hy4 may be more compelling.
I wrote about this a couple of weeks ago. It's actually often the biggest cost and it tends to be hidden away on most platforms!
https://martinalderson.com/posts/watch-out-for-cache-read-co...
Btw I still haven't came across any decent model that is <$0.01/MTok cache costs apart from deepseek thru their official API (even with the price increases).
Seems like a bit of an opportunity for someone to take - drop cache read costs significantly.
For me, an average long session results in about 200-300M cached input, 4-800K input, 2-400K output. Mostly the lower bound. Output depends on how much the model thinks.
There are two problems here:
- cache hit pricing (both Muse Spark 1.2 Contributor and MiMo 2.5 are around the $0.002-3/M mark)
- cache persistence time
Muse Spark drops the cache in less than 5m. MiMo keeps it around for at least an hour based on my experience with whoever is serving it for OpenCode. This difference itself will inflate bills massively.
A 500K token input repeatedly read by MS 1.2 for full input price 12 times an hour = $0.60. You would be expecting $0.012. So a 50x difference. Same thing on MiMo 2.5 is $0.018 because of longer cache times.
even with the 5m cache, Muse Spark Contribs is still best bang for your buck for the intelligence you get.
it is basically the old dsv4-flash prices, but even more smart.
That's because Deepseek invented the paradigm of prompt caching, they are the SOTA when it comes these techniques. Despite them open sourcing all their research, nobody beats them.
edit: I do wish openrouter would let you sort providers by Cache Hit % and Cache cost. These are the only things that matter to me at this point when choosing a provider.
Cache hit % on openrouter is not a good metric, it's mainly driven by openrouter's own provider juggling than the providers themselves
9 replies →
You can click the table headers to sort Ascending/Descending.
3 replies →
>Deepseek invented the paradigm of prompt caching
Caching was always here, you don't need to do anything special to get it on a single user local backend running a base model or a chatbot in the first place. Among commercial providers, OpenAI adopted it in 4o first.
If you’re Tencent you can just plug it into some field somewhere that lots of people see right? Like how Meta could put their model on Instagram search
It's very likely tencent games those stats, buying their own tokens.
Which explains why almost none of my request go though
[flagged]
i'd be curious if openrouter is just being gamed by these publishers by paying for the exposure.
wouldn't trust they dont do Capitalism like the rest of the AI field.
Of course they are. Of course they do. Nobody should be surprised by this.
>dont do Capitalism like the rest of the AI field
Like lobbying the US president to harm their competitors?
I would suggest "lobbying" is not the correct word to describe all the corruption going on in the current USA administration cesspool.
5 replies →