Comment by skybrian
1 day ago
There's a good reason to look at it separately: if inference is profitable then they make money (or at least lose less money) when they get more customers, because any fixed costs are spread across more usage.
1 day ago
There's a good reason to look at it separately: if inference is profitable then they make money (or at least lose less money) when they get more customers, because any fixed costs are spread across more usage.
Assuming that there are infinite suckers with cash to spend. It's entirely possible (if unlikely) that the market is not big enough to cover the training costs. Especially for multiple companies all burning insane amount of money on the regular.
Depreciation is part of the cost of inference. Inference happens in GPUs that have a relatively short lifespan.
Those GPUs are very expensive.
Inference is expensive because a GPU can only process a certain amount of requests in a given timeframe. Remember that Anthropic is constrained in compute.
If they are constrained, it means that those GPUs are not idle. If they have more customers, they will need more GPUs.
If they have to play silly games using EBITDA to be "profitable", then it means that they need to ramp up prices a lot more than they already did.
Which is why in these discussions I always say that inference is also extremely expensive. Too many people like to pretend without any evidence that inference is cheap.
Anthropic and OpenAI don't own data centers. Since they're renting GPU's, that's not depreciation. Paying rent is an operating cost.
Language models don't wear out the same way; upgrading is a choice.
Not a real choice.
You can "just not update an LLM" in theory. But if your competition updates LLMs, and gets more capable, more efficient LLMs, and you don't? They get more capable "expensive tiers", and cheaper "cheap tiers" of LLMs. What are you going to do then? Bleed userbase and die?
1 reply →
If they’ve entered into contractual agreements then actually it is debt.
I think the key thing that depreciates is all their models. You train one at crazy cost and 6 months later it’s worth $0. If you ignore that depreciation you look much more profitable.
Model inference compute outweights training compute by 10:1 and more for frontier LLMs. "LLM depreciation" is an expense, but not a dealbreaker.