Comment by bandrami
10 hours ago
Nope. They're losing money on straight inference (you may be thinking of the interview where Dario described a hypothetical company that was positive margin). The only way they can make it look like they're making money on inference is by calling the ongoing reinforcement training of the currently-served model a capital rather than operational expense, which is both absurd and will absolutely not work for an IPO.
Inference, in and of itself, can't be completely unprofitable. Unless you're purely talking about Anthropic?
But
> If you want LLMs to continue to be offered we have to get to a point where the providers are taking in more money than they are spending hosting them
Suggests you just mean in general, as a category, every provider is taking a loss. That seems implausible. Every provider on OpenRouter is giving away inference at a loss? For what purpose?
Do you have sources? I would be interested to read them
Probably the best roundup is Ed Zitron at https://wheresyoured.at
Half the articles are paywalled but the free ones outline the financial situation of the SOTA providers and he has receipts