Comment by bandrami
14 hours ago
If you want LLMs to continue to be offered we have to get to a point where the providers are taking in more money than they are spending hosting them. And we still aren't there (or even close).
14 hours ago
If you want LLMs to continue to be offered we have to get to a point where the providers are taking in more money than they are spending hosting them. And we still aren't there (or even close).
They are taking in more than they are spending hosting them. However, the cost for training the next generation of models is not covered.
Nope. They're losing money on straight inference (you may be thinking of the interview where Dario described a hypothetical company that was positive margin). The only way they can make it look like they're making money on inference is by calling the ongoing reinforcement training of the currently-served model a capital rather than operational expense, which is both absurd and will absolutely not work for an IPO.
Inference, in and of itself, can't be completely unprofitable. Unless you're purely talking about Anthropic?
But
> If you want LLMs to continue to be offered we have to get to a point where the providers are taking in more money than they are spending hosting them
Suggests you just mean in general, as a category, every provider is taking a loss. That seems implausible. Every provider on OpenRouter is giving away inference at a loss? For what purpose?
Do you have sources? I would be interested to read them
1 reply →
The open models may not be as great but maybe these are good enough. AI users can switch when the prices rise before it becomes sustainable for (some) of the large LLM providers.
Currently it costs so much more to host an open model than it costs to subscribe to a much better hosted model. Which suggests it’s being massively subsidised still.
For a lot of tasks smaller models work fine, though. Nowadays the problem is less model quality/speed, but more that it's a bit annoying to mix it in one workflow, with easy switching.
I'm currently making an effort to switch to local for stuff that can be local - initially stand alone tasks, longer term a nice harness for mixing. One example would be OCR/image description - I have hooks from dired to throw an image to local translategemma 27b which extracts the text, translates it to english, as necessary, adds a picture description, and - if it feels like - extra context. Works perfectly fine on my macbook.
Another example would be generating documentation - local qwen3 coder with a 256k context window does a great job at going through a codebase to check what is and isn't documented, and prepare a draft. I still replace pretty much all of the text - but it's good at collecting the technical details.
1 reply →
Efficiency goes way up with concurrent requests, so not necessarily subsidy, could just be economy of scale.
You can use open models through OpenRouter, but if you want good open models they’re actually pretty expensive fairly quickly as well.
2 replies →
If I drop $10k on a souped-up Mac Studio, can that run a competent open-source model for OpenClaw?
5 replies →
It is nobody's responsibility to ensure billion dollar companies are profitable. Use them until local models are good enough
I think this has to be done with technological advances that makes things cheaper, not charging more.
I understand why they have to charge more, but not many are gonna be able to afford even $100 a month, and that doesn't seem to be sufficient.
It has to come with some combination of better algorithms or better hardware.
Making it more affordable would be very bad news for Amazon, who are now counting on $100B in new spending from OpenAI over the next 10 years.
Somethings not adding up. Why is Amazon making financial plans for the next decade based on continued OpenAI spending but you’re saying AI providers like OpenAI and Anthropic aren’t even close to being profitable, so how can they last a decade or more?
Who’s wrong?
4 replies →
Someone's going to get burned here that's for sure. This isn't going to end with every person on the planet paying $100 a month for an LLM.
1 reply →
They probably aren’t planning on making the money on consumer subscriptions. Any price is viable as long as the user can get more value out of it than they spend.
"Sell this for less than it cost us" was a viable business plan during the ZIRP era but is not now
I'll take local models over these corporate ones any day of the week. Hopefully it's only a matter of time
If they started doing caching properly and using proper sunrooms for that they'd have a better chance with that
If my empty plate had a pizza on it it would be a good lunch
I see the current situation as a plus. I get SOTA models for dumping prices. And once the public providers go up with their pricing, I will be able to switch to local AI because open models have improved so much.
Like with all new products. It takes time to let the market do its work. See if from a positive side. The demand for more and faster and bigger hardware is finally back after 15 years of dormancy. Finally we can see 128gb default memory or 64gb videocards in 2 years from now.
[dead]