Comment by f311a
5 hours ago
Opencode said they are working on matching the old prices using their own inference.
Right now, they give 4100 credits for Luna and 63 000 for Deepseek on their prepaid plan (both are 2x)
5 hours ago
Opencode said they are working on matching the old prices using their own inference.
Right now, they give 4100 credits for Luna and 63 000 for Deepseek on their prepaid plan (both are 2x)
How could that possibly work? Deepseek was undercutting every other provider by an order of magnitude on cached tokens.
Do they just set a super low caching time and hope that drops effective cache rates low enough? Do all other providers somehow overcharge by that much? Are they just going to sell it as a loss leader?
> Do all other providers somehow overcharge by that much?
This, I think. Cached inputs have an opportunity cost (keeping the KV cache until use) but a hit is basically free. “Basically” - if the cache is offloaded to system RAM or NVMe there’s some scheduling overhead.
From a consumer viewpoint a more interesting metric than the raw costs is
from a personal standpoint rather than a per-provider one (e.g. if OpenRouter is blindly dispatching your requests you might have a bad time).
> if OpenRouter is blindly dispatching your requests
This can somewhat be the case, depending on your config. I updated mine to make DeepSeek high priority because I was having a lot of cache misses and reliability issues with the default (cheapest (at face value)) providers, and cost was actually higher overall than anticipated. Was smooth sailing from then; might have to tweak things again now pricing has changed though.
1 reply →
System RAM and/or NVMe storage still has a real cost. And swapping out the context between VRAM and system RAM / NVMe still consumes bandwidth.
I don't have a clue on what the real cost to inference providers comes out to, but it seems really weird that there would be such a big gap, in what should be a pretty competitive market.
1 reply →
I doubt they will match old cache read pricing- that’s most important in agentic coding.
CXL might save us. All the world's old DDR4 to the rescue. Either per box, where the job has to route back, or network attached, where there's now a pool of absurdly fast temp storage.