Comment by panarky
17 hours ago
> no way that is remotely close to the actual cost
Anthropic is running 85% gross margins on inference.
17 hours ago
> no way that is remotely close to the actual cost
Anthropic is running 85% gross margins on inference.
Yeah, on the API, not subscriptions. Those are a loss leader
No, inference as whole, including all the subscriptions.
The API Gross margin is even higher.
Fair. Perhaps it would be better to say a fully utilized subscription is a loss leader
1 reply →
Less development and operating costs
Yeah and soon we'll discover it's because they calculate the cost of inference as near 0 because they use AWS / NVidia / etc. credits that they got "for free" (aka investments from those companies) instead of paying for them, or some other accounting bullshit.
You wanna know the real price? It's easy, just check what AWS, GCP or Azure charge for tokens.
OpenRouter gives you easier to verify cost analysis of generic inference, as you can know the model weight size and so on, and see how competing providers price that
[flagged]
No, for a dollar of inference sold, (as an average of all the different inference tiers and plans), it cost them 15c to provide it.
This excludes the R&D costs, including training, where most of their costs lie.
The basic truth of LLM economics is that inference is cheap. The NRE cost to get where you can get premium prices for inference is sky-high. This is part of why the AI companies want to drive the scale up so much.