Comment by abound
18 hours ago
It's really refreshing to put some cold hard numbers on this stuff, hopefully this quells some of the "AI is unprofitable everyone is spending a dollar to make fifty cents" arguments. Amortized cost of owning a DC full of GPUs is far less than $38/card/hr
Not to say it can't still be a bubble, just that there is legitimately profit to be had and perhaps some sustainable equilibrium in the future.
> hopefully this quells some of the "AI is unprofitable everyone is spending a dollar to make fifty cents" arguments.
I think everyone has known that inference is profitable overall for the major providers for a while (I'm sure not all user accounts, but in aggregate).
The major providers are unprofitable after you add in the training and research costs and other overheads.
This is why the open source models are such a threat if they become good enough - it will destroy the business model of just providing raw chat/intelligence.
Arguably that's also why a few of these providers are trying to diversify into whole-platform approaches rather than just relying on having their whole thing be a great model.
There has always been a lot of evidence that prices are only going down. The fact that you could already get DeepSeek or GLM inference from 20+ providers on openrouter.ai was already sufficient evidence that inference was profitable and being commoditized.
People that want to believe something different as a coping mechanism will not be swayed by this evidence either.
AI will end up being a commodity. That's also why you already see the first rounds of protectionism under the guise of national security. Obviously they do have a point but that point works against them just as easily. Anybody that integrates unreviewed model output into their workflow is asking for trouble.
49 replies →
I’ve never heard anyone say raw inference was unprofitable. It wouldn’t even make sense, I can’t see a rug pull for renting out commodity GPUs.
People are saying _training + inference_ is unprofitable (I don’t have a dog in the race and don’t really care either way). That’s a lot of GPUs and salaries that aren’t generating revenue at any moment.
Personally, I would guess Anthropic and OAI are profitable. I just suspect that they’re a pretty normal level of profitable and not the absurd, near-alchemical “can turn dirt into gold” level of profitable their valuation seems to have priced in.
4 replies →
It's a commodity, and research into how to make it good will be done in the open. What this means is that we should doubt the justifications offered by magnificent-N-whatever companies slurping up all investment dollars to take over the world by being the first to get "general" "super" intelligence. No, there won't be a runaway improvement. Sadly, AI is just a clone of our inner world. Getting interesting information into it is still going to be just as slow as it is in the regular world. You have to build, tinker, experiment, get data, get knowledge, etc. The only thing getting sped up is how much time needs to be allocated to thinking and translating.
2 replies →
There may be different thesis out there, but the fact of prices going down definitely doesn't help the case that the major AI labs are are overvalued and need to vastly increase their revenue to stay afloat amidst their contractual obligations.
I honestly don't think there are too many people arguing "AI is actually too expensive to take my job". "AI being too much of a commodity to make meaningful profits" could very well be the case though.
1 reply →
> I think everyone has known that inference is profitable overall for the major providers
> The major providers are unprofitable after you add in the training and research costs and other overheads
It's meaningless to say that selling something is profitable if you are ignoring the cost to produce it.
The profitabiity (or not) of AI models needs to be calculated on a per model lifetime basis - what did the model cost to develop and train, what is the lifetime revenue the model generates, and what is the lifetime operational cost of the model (including amortized GPU cost), etc.
It makes no more sense to calculate "inference" profitability when ignoring training cost than it would to calculate it when ignoring GPU cost.
Exactly. And you can't just stop training models, because your competitors won't, and customers are switching providers on a daily basis (most likely an exaggeration).
Hence "we need to slow down progress for safety" [narrator: it had nothing to do with safety]. The industry is begging for the regulation so that there is a coordinated slow down and consequently move towards profitability.
1 reply →
> It's meaningless to say that selling something is profitable if you are ignoring the cost to produce it.
At the same time, training costs don’t scale with usage, so no matter how expensive they are, they can be paid for as long as inference is profitable. It is only inherently unsustainable if you lose money on inference.
Of course, it could be that the inference usage required to pay for the training is too high to be possible, but that is a separate argument.
1 reply →
> It's meaningless to say that selling something is profitable if you are ignoring the cost to produce it.
Don't enter that rabbit hole, you'd be surprised at how questionable are some accounting rules and practices, even in publicly traded companies.
if i can run an existing open model because its good enough for what most people need, I can just spin up an inference business, and put my spend either on trying to optimize it further or on customer experience
... and not worry about training cost at all.
a lot of the cloud businesses of the last decades have relied heavily on open source code, again skipping most of the cost to make the product/service being sold.
1 reply →
> The major providers are unprofitable after you add in the training and research costs and other overheads
The very short lives of all these models also factor in.
I think the funny thing is the more inference, the more tokens a specific version of a model spits out, actually does make it more profitable.
But somehow a lot of people truly think that each token generated increases the cost of OpenAI/Anthropic/... and shout bloody murder about "subsidies"
And I'm certain there are people out there that actually think the API prices of OpenAI and Anthropic somehow equal their electricity costs.
Nobody meets expectations of a 2 trillion dollar IPO selling a fungible commodity like tokens for old models.
And that's the rub - it doesn't matter if inference is profitable, the US labs have promised the whole kitchen sink, and taken serious investment on that pitch. They won't make it back in a cut-throat market for inference. They have to continue pitching the idea that only their models are worth using, for quality/safety/performance/etc reasons.
Which means continuing to develop and release new models. At huge expense.
Side note - while deepseek seems to be doing fine, I'll note that the emphasis on cost effectiveness is very much not a thing for the US labs right now... and it was for non-us labs. I'd be quite interested in knowing if anthropic is pulling in the same returns, or what the margin on their subscription plans actually is.
---
I'm already seeing this directly at work - lots of devs still happily use claude for development, but the products we're making aren't. They'll use kimi, glm, gpt-oss, etc... basically - if cloudflare is hosting it, we're exploring it (mainly for contractual/security reasons): https://developers.cloudflare.com/workers-ai/models/.
Because we are price sensitive, and we already have to do all the work of creating very robust evaluation frameworks for our products, so we might as well choose the cheapest model that meets our standards.
whiiiiich is why we're seeing things like Haiku and gpt-luna priced down near the open models at ~.10/mm in, ~.50/mm out. They've been forced to cut prices to keep commercial customers on their cheaper models instead of moving away.
This will continue. If inference is profitable and easy, prices will continue to erode. Go browse openrouter, plenty of providers happy to undercut the competition. Bigger shops with auditing requirements will avoid them, but smaller ones won't.
20 replies →
>I think everyone has known that inference is profitable overall for the major providers for a while
Have you seen any place that is not as tech-savy as HN? e.g. some Reddit subreddits?
Barely anyone is aware other than people who are already somewhat deep into this whole topic.
The insider baseball in AI is extreme I agree. I still know people who insist large language models are just glorified autocomplete.
1 reply →
Yes. I've always suspected inference is profitable.
But I'm also interested is retraining needed? how relevant will a model be in N years if it's just "static", and if you do need to retrain and fine-tune over time, is it still profitable? how often is that needed?
For most cases it's probably ok, often times answers are general or basically a web search anyways, but if my Agent started blasting outdated syntax it would suddenly be way worse for coding for example.
It’s helpful for a model to be somewhat static when it’s involved in business processes.
Relying on an LLM's baked-in syntax knowledge is suboptimal, compared to leveraging reasonable context affordances (skills, examples, tool calls, MCP servers, search) to ensure adherence to precise APIs and language specs. Don't get me wrong; a model specifically trained on your desired framework is obviously preferable for more consistent generation of idiomatic code -- but relying on that in a large general-purpose model is a questionable proposition IMHO.
They need to be retrained often is my experience for general usage.
I have SO many cases, for example when shopping, when they make assumptions based on their cutoff date, and they need to redo everything based on lots of searches.
Sure, for many use cases old models will be useful but for general usage a model that's 5 years old will probably be garbage for most people, unless they prompt it extensively, which the general public won't do.
> But I'm also interested is retraining needed? how relevant will a model be in N years if it's just "static", and if you do need to retrain and fine-tune over time, is it still profitable? how often is that needed?
Just wait until what Ilya Sutskever has for Safe Super Intelligence (SSI) to self update the weights without retraining them to bring training costs down close to $0.
> I think everyone has known that inference is profitable overall for the major providers for a while
A lot of AI haters like Ed Zitron and their readers have refused to update on this. I don't know if any of them have yet, but it'll take a year or two for the idea to fully filter out of their podcasts anyway.
Seems a bit of a disingenuous framing. Most of the arguments I see are some mix of
1. Their figures have been pretty dodgy or excluded training costs so why is it trustworthy this time?
2. API inference charged at a profitable level rather than heavily subsidised subscriptions means everyone tightens up their usage + governance Which we very much saw with the rise of openrouter et al.
Personally I think there's a lot of the bait and switch of 2 when people talk about profitability of the 10-20% of API usage for Anthropic while also wanting to lump in subsidised chat users as if they're profitable too.
I think we know if it was this clear cut then the behaviours of companies would be wildly different and more consistent with profitability.
Ed Zitron is a professional apocalypse predictor. Which is to say he's always wrong.
It’s not training and research that kills you here, it’s your own investors ROI expectations that murder you cold when faced with Open Source commoditizing you, from a country that spent 5% of the sum you spent to get to the same position and then gives the recipe away to everyone.
> I think everyone has known that inference is profitable overall for the major providers for a while
Unfortunately while in the tech industry some might know that, the general public are still being misled (especially by certain folks who’ve made a living off downplaying AI) to believe inference itself is unprofitable.
> The major providers are unprofitable after you add in the training and research costs and other overheads.
This is correct for now, but I recently read that 90% of Anthropic's code is written by Claude Code itself. So, if an AI is using its previous version to train the next version, we're not far from the day where these models will take up the decision making process too. For eg: Decisions API by OpenAI or Jev is based exactly on this. What if a model like Jev sits behind Opus 5.5 and guides it to take the next step? Wooh.
Probabilistic predictions right now are at 99% for frontier models. They will never be 100%, but if humans rate themselves 99.9% in decision making, that benchmark can be broken.
As far as I know they haven't been doing any mass layoffs.
So if Claude Code is writing 90% of the code, then why are they keeping probably the most expensive engineers in the world on the payroll?
The machine writing all the code seems to not be enough, even for the company with access to the best models. What's the alternative to explain this discrepancy?
3 replies →
Yeah I think one of the less considered costs when people talk about this is the human cost of training. We all talk about the billions spent on compute, but I don't think anyone's got good numbers on how much RLHF is costing but it's got to be pretty large.
As someone who has long bought into the idea that the labs are not profitable on inference, I have a question. How is it simultaneously true that there is more demand for inference than supply (apparently by orders of magnitude), but also that inference is so cheap that labs can make healthy margins? It seems like the inference providers, not the model providers, should have the leverage. Why haven’t they said, “give me a bigger cut of each token or I won’t serve your model”?
Also, what happens when the inference providers suffocate on their own debt burdens? Who will serve inference?
It's a few things I assume:
1. Inference providers are largely a commodity business. While demand outstrips supply, there are many providers of that supply, which does somewhat constrain how much leverage a provider has.
2. The models are not interchangeable as of yet (though this seems to be changing). A lot of people want the latest Anthropic or OpenAI model; as such, the labs still have considerable leverage.
3. A lot of the inference providers have used their leverage by doing equity deals with the labs, agreements to have some of the deal revenue spent on compute, etc.
3 replies →
The only moat is in datacenters. The software has no moat, since 1) it's already trained on stolen data, and 2) someone with access to AI will be able to reverse engineer a proprietary implementation eventually, the same AI is being used for developing AI.
That's why so many talks about regulations. Regulations means: ban outside competition, just like US bans Chinese EVs today.
regulation is braindead
I would classify training and research more as capital investment than overhead.
Most people in my experience refuse to believe it for some reason.
Well, as that means there's lots of money to be made lowering costs for research and training! What tools have we used in the past to lower those costs? Automation.. Hm, what if we had AI completely automate AI research? Uh oh, we just rationally found outselves in a trajectory where we lose control of our own trajectory because of economic incentives.
It is wild how fast we're hurtling towards this outcome, lmao.
I doubt that, I got downvoted several times pointing out inferencing is insanely profitable.
Even take account of R&D cost it is still profitable, because you do not actual retrain the base model every month, only distill and post train from the same foundation.
heck, I think even including the giant cost of the top tier researcher salaries, they are still profitable.
then, how are they still losing money? just read what they are actually spend the most budget: always the hardware. those companies will never profitable, because they will never stop spending money to expand their computational prowess.
If the frontier labs are struggling under the training costs and the leading open source models are making their progress by distilling from the frontier labs it sounds a bit like there’s something wrong with the economics here or that progress is going to slow quite a bit without the amount of investment.
I don't think so. The number of comments I see from people talking about AI, there are plenty who still think the price is going to shoot up. Some even talk this way about models you can run locally on high end yet still consumer level hardware.
I think the more someone works with AI the more likely they realize that inference does not have to be sold at a loss, but many people's AI knowledge seems quite dated and it sounds almost like it becomes a convenient conspiracy theory for some, a way to convince themselves that the market changes that are already happening a temporary (and I don't mean in a Jevon's paradox sort of way, but that companies will go away from AI).
I suspect a lot of this is fueled by a few stories of companies who went all in on spending as much as possible on AI and then begun to walk that back, without realizing that was more about bad incentives and over promises of what AI could do (especially 1/2 model generations ago). Some also confuse running the SotA models at the fullest capacity costing more and more with a given problem costing more and more to solve, not realizing that the cost per problems is trending down but newer and larger problems can be solved for even more, so the costs at the top are rising despite falling prices. Also things like the subscriptions getting more expensive, though how much of that is because developers are doing far more than they were 6 or 12 months ago (and in some cases there was subsidization that is going away). And maybe a bit with the free models starting to not be upgraded or the harnesses getting worse performance of lowered free limits.
A lot of people have already replied to this point but I think I want to distill the problems a little more directly:
- If inference is this profitable, is it profitable enough to offset the enormous training costs, bot those already incurred and those in the future?
- If inference is profitable, can it cover other capital and operational expenditures the hyperscalers and neoclouds are committing to, on the timelines they're committing to, to scale out inference?
- If inference is profitable we can measure if the price/earnings ratio of current valuations is sane. Is it?
- If inference is profitable, is it - given the answers to the questions above - profitable enough to make the risks of lenders of the huge amount of debt companies are taking on to scale, rational? It's one thing to say as a hyperscaler "we can make this pay", it's another to say "I'm going to lend them the money to do that and I'm going to get my money back on the timescales we agreed, and with an ROI to boot".
- Finally, if inference is profitable why is Nvidia specifically lending money to companies to buy Nvidia cards to do that inference, and then hitting the credit default swap markets to cover the risk of never getting that money back, and doing so at ever increasing cost, when they could just go build a profitable inference business themselves?
I agree seeing the numbers is good. It does suggest profitability in some segment of the market. However I still have a lot of questions.
But you are mixing up different categories.
Most of your questions are things that are relevant to everybody due to a possible economic crash. And you are right to point this out.
But for me as an AI user those aspects are not really relevant in my decisions. If an AI bubble bursts providers will still want to sell me those tokens at that price and yes development will slow down for a time which does change not a lot for me.
Inference is very profitible when you can deal with the batch sizes Anthropic, Deepseek and OpenAI are dealing with, renting GPUs for inference is even more profitible.
The US is dealing with different tax code where buying GPUs function as as great way to offset taxes, and having boatloads of debt is rewarded in many cases. I think there's lots more (crooked) reasons why inference is even more profitible in the US.
100% the right questions. The implied thought process of so many on this thread: Inference profitable >> Frontier Labs' are profitable overall. Sigh...
I can't read the original Chinese text, but if it's revenue, then you still can't say anything about profitability. Housing and powering one GPU should be way less than $38 per hour, so there should be a profit on inference. If that profit can cover training new models we don't know, but it would appear to not be the case.
For the past year or two I think it has generally been agreed that inference is profitable, but the LLM business as a whole is not.
I seem to recall the leaked Liang Wenfeng speech gave a figure of 8-12(?) months of inference recouping the training costs of the model.
I think what he said was that 10 months of inference paid for the hardware it runs on (which may have a lifetime of a few years). I don't think he mentioned training/development cost.
1 reply →
I assume they need these models to stop being replaced in 1-2 months though.
It's really hard to unpick, for example, energy prices might be heavily subsidized for the Chinese models.
But it's clear that it's not the money burning pit I think it was just a year ago.
13 replies →
Would that be due to the size of the models from DeepSeek or perhaps only for their distilled models? If it's for general purpose models that would crash the valuation of Anthropic and OpenAI, both who are struggling with profitabilitet to say the least.
9 replies →
kind of on track with gemini's releases
google is a company that tries to make a point of having profitable subdivisions
What about sourcing the asset itself? It's like a factory is it not?
You pay for the asset (tenancy in a data center) and then you pay for housing, powering, cooling, etc.
I think it’s pretty clear from the cherry picked numbers at Anthropic.
Demand is the issue. Demand only spikes with a sota model release then in drops back down to unprofitable levels when factoring in training a new model and build out.
So your stuck in an unprofitable cycle of losing revenue when your not releasing sotas but losing profit when you are.
This goes back to googles original memo “there is no moat”.
Where in you can’t stop training sotas because you’ll lose customers to the next inference model above you instantly.
Google has seemingly defeated this by finding models cheap and good enough for their existing platforms resulting in a moat of distribution. Something OpenAI and Anthropic seems to lack.
Unless the money of the investors is going somewhere else...
I think you have to take this on a case by case basis - there is no reason to suppose that the training costs and serving costs are the same for all companies for all models and for all price plans. Some may be profitable, others not.
DeepSeek's models are relatively small, and training costs will be correspondingly lower than for larger models. All you have here is a revenue number - no mention of (training + serving) cost, so while they may well be profitable you can't determine that from this number alone (if it can be trusted).
DeepSeek (investor meeting leak) have previously said that they price their models at a point where the GPU is paid for after 10 months, so something doesn't seem to add up here. If they were making $38/hr per card 24x7, then in 10 months they'd be making $270K per card, while the cards (Huawei?) only cost around $20K.
Presumably cards run on electricity?
That turns out to only be a small part of the cost.
Say a $30,000 GPU has an operational lifetime of 3 years, and draws 1000W. If it is running 24x7 then over those 3 years it will consume 26,280 kWh of electricity, at a cost of say $0.10 per kWh, giving a lifetime electricity cost of $2600, or less than 10% of the cost of the GPU itself.
So, to a first approximation the cost of serving a model is the amortized GPU cost.
Now that the unit economics are laid out in the open, What stops any of the companies or even startups from anywhere in the world with some investment $ and competence from spinning up and offering their own version of the inference engines, undercutting any moat that the current leaders have?
I could even see F1000 companies having their own backstop intelligence for most of the tasks and only use frontier where it is absolutely needed or as much as it is under their IT budget.
Won't it be race to the bottom when they compete on cost? Is regulatory capture the only moat left for the market leaders now?
Nothing but hardware access, which is getting harder for the average person but for a giant corporation I see no reason they can’t compete now at the local model level
I don't think i'm paying more than Deepseek price, ever
Until the Chinese stuff came online I was completely convinced this is just a bubble waiting to pop and nothing will come of this tech due to its absurd costs. The Chinese stuff changed this picture massively. I'm daily driving DeepSeek 4.1 flash (at API prices) and every now and then I give Opus 5.5 a try and don't notice anything missing for me.
My workflow is one where I am heavily involved in the process and I don't like giving the agent too much autonomy + I need to follow along. Opus 5.5 speak is still cause aneurysms for my brain. It also has this vibe to its prose... like it's looking down on me. This is Prolly my bias speaking though.
> My workflow is one where I am heavily involved in the process and I don't like giving the agent too much autonomy
For such workflows it barely matters which of a dozen models gets used.
Whereas if you're handing off all dev and no longer reading code, just making CTO-level architecture decisions, then there are only 2~3 models right now worth using, none cheap.
> Whereas if you're handing off all dev and no longer reading code, just making CTO-level architecture decisions, then there are only 2~3 models right now worth using, none cheap.
I use frontier models every day and night, but not sure if the described approach is possible with any of them right now. Models still make stupid mistakes and people who do not read the code produced are bound for surprising discoveries in their codebase. Not trying to downplay the effectiveness of LLM-assisted coding and all that, but it's hard to imagine how much stupid shit i would've committed if I was not reading their output.
4 replies →
> I'm daily driving DeepSeek 4.1 flash (at API prices) and every now and then I give Opus 5.5 a try and don't notice anything missing for me.
I was doing the same (along with GLM 5.3 Flash) for months, and then gave Claude Sonnet 5.5 at medium effort a try. The difference was like night and day. Where both DeepSeek 4.1 Flash and GLM 5.3 Flash kept making mistakes and would have to do a bunch of iterations to find and fix them, Claude Sonnet 5.5 just did it right on the first try.
In my experience, while DeepSeek 4.1 Flash and GLM 5.3 Flash are a lot cheaper per token, they're just as or maybe even more expensive when you factor in all the extra tokens that are need to troubleshoot and fix their mistakes -- and even more expensive if I factor in my own time wasted doing the same, not to mention the frustration of having to deal with that.
DeepSeek 4.1 Flash also hallucinates a lot.[1]
I'm hoping all this improves with time, and the gap between the competent frontier models and open weight models narrows.
[1] - https://www.youtube.com/watch?v=P4dTq4X8bqk
I don't think it's a fair comparison. Maybe against deepseek pro would be more similar in terms of model size
Same for me. The cheap Claude subscription is just so much cheaper. Never had to buy more deepseek tokens for months (August), because Sonnet 5.5 is good enough
That’s the Model Welfare speaking, not your bias.
OpenAI's entire business is trivially profitable when you attach an ad network to ChatGPT.
Quick, someone point out the one billion user system with intent + personal data rich that doesn't result in a $100+ billion high margin ad network.
It is that simple and absolutely nothing can stop the gigantic ad networks that will be built up around personal agents.
Only if CPM can be high enough to counter balance the inference costs.
Current CPM is between $30 and $60 which is x20 to x30 more expensive than alternatives.
Now we have to wait and see if performance is also at least 30x of previous ad targeting.
2 replies →
The thing that stops it is, ironically, personal agents. Get one to interact with the ad-driven one if you need to and filter ads out.
1 reply →
What? Ad networks REQUIRE a moat. Google Meta Amazon have historically defensible moats. ChatGPT? Zero moat. Good luck with that ad business!
1 reply →
[dead]
But haven't the Chinese (and Deepseek specifically) put a lot of emphasis on running models efficiently (because they've been locked out of being able to obtain certain GPUs by the US)? Whereas the US frontier companies seemed to get huge cash inflows and didn't need to worry about efficiency as much and put more emphasis on frontier performance. So how do you know that this $38/card/hr is about the same that the US frontier companies are making?
I still feel like there are some questions there, as in what is the actual amortized cost, plus the ongoing variable costs and even the non-training related overhead.
Plus there is the actual demand side to consider and how does it impact when paid utilization isn't 100%.
I don’t know, I take this as there being a good amount of margin that is going to be suppressed due competition in the very near term. It will be interesting to see how these margins hold up over the next 12 months, I expect them to drop substantially
> Amortized cost of owning a DC full of GPUs is far less than $38/card/hr
You missed the bit where they used the word "revenue".
I suspect if you factor in all the various overheads of running datacentres the actual value is lower. Revenue is vanity, profit is sanity as the old saying goes.
However the Chinese do have the benefit that China are pushing hard on renewable energies. So at least that major part of the equation is probably far lower than your average US/EU site.
Huge, generational mistake for the USA to be leaning so hard into fossil fuels right now and actively trying to prevent buildout of solar and wind power. That decision is just going to hurt more and more in the coming years as China's electricity costs continue to plummet while the US' continues to depend on unsustainable fossil fuel infrastructure.
> actively trying to prevent buildout of solar and wind power.
And actually paying people not to ! (Paying RWE $1.2bn to abandon wind contracts[1])
Its absolutely nuts. Pure generational can-kicking.
[1] https://www.bbc.co.uk/news/articles/c1e1vg0gjl5o
> Huge, generational mistake for the USA to be leaning so hard into fossil fuels right now and actively trying to prevent buildout of solar and wind power.
If the US is trying that, they are failing pretty hard - 88% of new power in the US in 2025 was renewables, and 2026 is estimated to be above 90%
One of countless mistakes made by the current administration. Decimating research funding and firing many scientists is another.
3 replies →
Why? AI DCs can't rely on solar or wind so there must always be a thermal backstop. China is building lots of coal and nuclear.
Reality: renewables rapidly increasing in the US every year, especially solar (around 30%)
It's hilarious and depressing that one of Trump's supposed goals was making the US into an industrial powerhouse again, and yet he's acting to protect the profits of the fossil fuel industry as opposed to encouraging the kind of resource efficiency that would make the US competitive long term.
1 reply →
China is investing in cheap and secure energy, which includes renewables, but it also include coal: https://www.reuters.com/business/energy/chinas-coal-power-ri...
The latter is great as a way of getting an edge in business and reducing reliance on oil supplies, but its also environmentally highly damaging.
> You missed the bit where they used the word "revenue".
No they didn't...
> I suspect if you factor in all the various overheads of running datacentres the actual value is lower.
That's literally exactly what the quoted part addresses: "Amortized cost of owning a DC full of GPUs is far less than $38/card/hr". Cost of running GPUs is far less than $38/hour, revenue is $38/hour, ??? , profit
The railroads went bust, twice, not because there wasn't profit available, but because investment outpaced profit and wasn't sustainable.
> Amortized cost of owning a DC full of GPUs is far less than $38/card/hr
what do you think it is, out of curiosity?
Other replies covered this, but I was basing that comment on the prices that commodity providers charge for GPU-accelerated VMs. E.g. the going rate for a NVIDIA B300 (288GB RAM) and the VM around it is somewhere between $5 and $20 an hour [1].
These providers have very little incentive to charge unprofitable prices (except maybe capturing more of the market, but there's not much lock-in here), so it's reasonable to assume somewhere in that spread is the "true" cost of being in this business, all expenses included.
[1] I'm using https://gpus.io as a reference here, and that's on-demand prices, which will tend to be the highest relative to other options.
(tongue mostly in cheek) sounds like I should be renting out these platforms and selling inference on them to OpenRouter
You can pay off a high end Nvidia DGX server at just over $7/hr/card over a 1 year period.
$38/card/hr is a ton of money. That's $2.6 million per 8-card server per year.
The cost of power/physical space/interconnect by comparison here is negligible.
...well you can get B200 in cloud for ~$6-14/hour and that's after all hosting costs and profits company make of it soooo yeah
With this new data at hand, I have to wonder how profitable would it be for the investors.
The cost of amortising a datcenter, the gpus, the engineering costs attached all to produce inference.
At how much percentage of its inference would it be profitable (if it does), how much returns would it make over the long term and what would be the profit of those inferences/ profit-revenue and how that compares to other industries.
Are there any statistics which show this. Also how efficient token inference has gotten over time?
https://inferencex.semianalysis.com/profit-estimator
Those arguments will go away when spending on AI infrastructure is no longer some multiple of gross annual software sales revenue, yeah?
When that happens, the chips stocks will have tanked back to sane levels and the doom posting will be about that. That will put us in the 2001 era of post dot com bubble, and the following few years will show us who the winner are
>It's really refreshing to put some cold hard numbers on this stuff, hopefully this quells some of the "AI is unprofitable everyone is spending a dollar to make fifty cents" arguments.
Inference has always been profitable. The problem is that this profit shrinks as your model becomes less competitive. The labs are in a constant race to release the best model to maintain inference profit. Deepseek is not training their own models at the scale of OpenAI and Anthropic and mostly sustains off their efforts via distillation.
You're looking for comfort in the wrong place.
From microns S10: BITS shipped and net revenue.
Q1 '20 - $0.10 profit per gb shipped (ram and storage)
Q1 '23 - -$0.024 (cyclic ram market)
Q1 '25 - $0.131 this looks normal were on the upswing.
Q4 '26 - $1.136 Were all fucked.
If you look at a chart of S3 prices we have all been living in the "Happy Time" where the cost to run our ever growing software business has gone down. Githb says 2025 billion commits - 14 billion this year, and 2x the actions run. That means AI driven software growth is 2x to 14x.
But those ram/storage prices are going to hit the operational budget of EVERYONE soon.
Can your average sas survive a 4x price increase on its AWS/Azure bill? How about 10x (cause thats what the margins say the premium is on ram).
If the circular financing is a concern, the circular firing squad should be a bigger one. Your paying for tokens to build more software to pay more to run your software is not a sustainable cycle. People are starting to use the words CapEx again - and it's a good thing.
And it seems we're getting just 10x more software nobody uses.
We don't need 10x more software per se. It's removing the high cost and exclusivity from software production (but not just software production). It also enables the production of highly specialized software by non-specialists, so it broadens the field dramatically (things that were not being produced, will be). Simply put, every bit of software that can be useful, will definitely be made; and a whole lot of unnecessary software will be made in that process as well, that's chaos-discovery for you.
> "AI is unprofitable everyone is spending a dollar to make fifty cents"
you are just talking about ai inference? Are you assuming ai models themselves fall from the sky.
> It's really refreshing to put some cold hard numbers on this stuff, hopefully this quells some of the "AI is unprofitable everyone is spending a dollar to make fifty cents" arguments. Amortized cost of owning a DC full of GPUs is far less than $38/card/hr
They (well, most of them) are looking at loans and debt (in plain sight and hidden) of AI companies and asking "what will pay for that", not looking at how much a GPU costs and how much tokens you can sell off it.
Just putting a model on a GPU and selling it is profitable, period.
But spending more and more money on building and running infra to build frontier models that solve smaller and smaller piece of the pie might very well be unprofitable.
Models competing with the frontier labs gets more capable every month and more and more of the tasks can be achieved without reaching for the big guns.
We might very well get to the point where AI/LLM is profitable just fine but bubble still bursts and the big frontier labs are in trouble. And frankly that would lead to far more sustainable ecosystem.
revenue != profit?
AI as a business will be 10x larger than it is today within the next 20 years.
Present valuations are the bubble, not the business / industry. The exact same thing was true about the Internet bubble circa the late 1990s, except this is a lot more important than the Internet and the global economy is drastically larger.
Productivity is how you expand economy and wealth. It's how China can go from being one of the world's most backwards agricultural nations, unable to feed itself properly, to gradually becoming an agriculture juggernaut. AI is a hyper productivity tool, saving people time and bolstering capability are two of the primary productivity concepts.
We're not even out of the first inning yet.
Why do you posit that this is a lot more important than the Internet?
Artificial intelligence will 'infect' every surface of life and business. The Internet is merely a distribution platform for intelligence (and yes that includes TikTok videos). Its first distribution chore was slow and quaint, it was human. The intelligence was always the higher value, not the distribution network.
Distribution of intelligence: data, photos, video, communications, transactions, entertainment, et al.
And to be clear, it doesn't matter if some random commentors on HN don't think very highly of some or all of the intelligence being distributed. Their elitist opinion of the quality of intelligence of a given thing doesn't matter as to whether it's happening or not.
When a 16 year old records a cloned TikTok dance video, edits it, and then publishes it, that's intelligence distribution. That's a person thinking through a series of actions to try to communicate to an end audience, via an entertaining medium. That's intelligence being distributed over the Internet. That's the quaint, slow, human form of it.
Up to this point that distribution was basically entirely human, capped (post saturation) at slow growth N intelligence output (derived from the total number of humans and their time / productivity capabilities). AI uncaps that and radically expands intelligence output & distribution at a growth rate that is so fast you might as well start picturing the planet having tens or hundreds of billions of people on it. The human population will shrink, the artificial population will increase dramatically. People will have more artificial friends than human friends within ~20-30 years.
A few decades from now the notion of thinking of computing separate from AI will be so foreign most people won't understand what you're talking about. It will just be thought of as computing. Humans will forget a time when we weren't augmented by the constant presence of artificial intelligence. Try taking 'smart' phones away from the masses. In about a decade try taking away their helper agents. A decade after that try taking away their home robots.
The steroid jump is AI + robotics. The Internet is trite compared to what comes out of that over the rest of this century. Robotics will physically remake our world, in a way the Internet can't. We'll deposit tens of billions of robots onto the surface of the planet and they will almost all be the result of having figured out how to make intelligence (relatively) easy & (relatively) cheap. I'm not talking about Roombas.
The Information Age is over. We're into the Age of Intelligence (insert the unavoidable quips about humanity & intelligence).
The Internet is just one way to move intelligence around (a robot body is another, a smartphone is another, a human body is another, and so on). It's a sub value to intelligence broadly. Intelligence is the ultimate value. The valuations (and sales growth) for OpenAI and Anthropic is because they're vast intelligence output machines in the Age of Intelligence, ditto Nvidia's business boom. We can now mass manufacture intelligence at an accelerating clip, and it has gotten cheaper at a comical rate over the past few years. While HN comments snort about pelican improvements day to day, the leap over a few years is astronomical in regards to what significant AI systems can now do (with LLMs being just one type of that).
15 replies →
> AI as a business will be 10x larger than it is today within the next 20 years.
I would argue it will be 10x in 2-3 years the way things are going.
Today. As new supply comes online and demand levels off, that 38$ will shrink. Absent legal monopolies, the margin will shrink to a sliver more similar to any other service. That is the danger of open source models. They destroy monopolies and shrink margins.
> That is the danger of open source models. They destroy monopolies and shrink margins.
Danger? You mean benefit?
That would depend on whether you are a user/customer or an investor/owner in AI. At the moment, nearly everyone is on the investor/owner side. The entire western world has doubled down on AI becoming the most popular product since the invention of fire. If it turns out that fire 2.0 cannot be patented/copyrighted/licensed for insane profits, those dollars disappear.
14 replies →
Depends if you’re a shareholder or not
> That is the danger of open source models. They destroy monopolies and shrink margins.
It's unclear that the leading open-source models could exist without having distilled the frontier models. If that's true, the "danger" is that these upstream models don't get built any more (because of the tremendous cost), and we end up just optimizing what we have. Also unclear whether this would be a net win or loss for humanity.
The tech is at the point where synthetic data is almost high enough quality to train further models on, AND there are so many human users of cloud AI, that their usage can be gathered and trained upon (it's human generated and used in real world situations, so, it is of a good enough quality). The cat is fully out of the bag.
2 replies →
The upstream models weren't built by god, they were built by distilling the world's knowledge and cultural products.
If OpenAI/Anthropic stop doing it there's no reason to believe someone else won't. Or that we won't come up with a different approach.
1 reply →
I'm OK with this.
Anthropic is zooming to $100 billion in annualized revenue.
The tremendous cost is matched by the tremendous business output.
In the span of less than five years, starting from their first $1 of revenue, Anthropic + OpenAI will have produced businesses doing at least $150 billion in combined sales. The demand isn't reducing at present, it's climbing, they can't come close to servicing the total demand for their frontier models.
SAP is under $40 billion in annualized revenue. They're 54 years old.
5 replies →
It is mostly response to claims of "all LLM are subsidized and blatant bubble, LLM are useless" etc
All LLM are subsidized and blatant bubble.
If the only profit is from running commodity free software (as OP claims), there's no moat and OpenAI/Anthropic are worthless.
you actually have no idea what the cost is
We still need to know if that includes purchase and maintenance costs, which are substantial. And then of course all other costs associated with it.
You also have to realize deepseeks running costs are immensely lower than say any us lab by orders of magnitude.
Chinas grid is almost entirely renewable energy with strong investments in it. In the us not the case, some dcs are running on gas.
Also DeepSeek was smart. They strategically rolled out model sizes they could support even if the intelligence wasn't the world's best for that week in human history. Then they have incrementally and cheaply improved models since.
Not to mention they also built trust with open weight models rather then fear mongering. They are in the top 3 of the smartest most viable LLM labs out there.
China generates more electricity from coal than from all renewables put together (49.7% per https://oilprice.com/Latest-Energy-News/World-News/Solar-Ove...).
They have more installed renewable capacity than coal, but renewables have low capacity factors, solar is less than 50% by definition. The renewable share is growing fairly rapidly, but it is by now means almost entirely renewable.
[dead]