Comment by Tuna-Fish
2 months ago
This is an example of common knowledge that is wrong. People look at their cash burn, assume that they spend this to subsidize inference, and get bonkers answers. Inference is not their largest expense.
Inference is cheap. Anthropic is only drastically subsidizing their plans if you count their training expenses as part of their costs.
If "inference is cheap," why is OpenAI spending a ton getting Broadcom to design custom AI chips that make inference cheaper? Reports suggest their custom silicon isn't all that good for training, it's all to make inference more efficient. That shouldn't be necessary if inference is already quite cheap.
A large part of the market will be ad based. For that, having the lowest cost inference is useful.
Also for agent doing r&d, cheaper tokens allows doing more, which is always good.
But the training expense is part of their costs!
The question is can they just stop training at some point, fixing the models in time, and still have a useful product.
Are you an anthropic insider or something? Because if you are you should delete this comment. If you aren’t then you don’t know what the hell you’re talking about.
For one point, you can look at the costs of similarly sized open-source models from inference providers (which are only making money on the markup on the compute), and compare with anthropic's prices. There's a pretty big price difference there and it would be hard to believe that anthropic's models are that much more expensive to run than those models.
Those prices don't need to bake in the training cost, since it was eaten by someone else (whomever trained the open source models). Anthropic et al. need to price in the whole lifecycle.
1 reply →
Surely the same can be said for the people saying the opposite?
I didn’t make a claim. The parent explicitly said it was a misconception that inference is not profitable.
No one knows if it’s profitable or not so we’re left to speculate.
3 replies →