Comment by ghc
6 months ago
> In order to break even they would have to minimize the operating costs (by throttling, maiming models etc.) and/or increase prices. This would be the reality check.
Is that true? Are they operating inference at a loss or are they incurring losses entirely on R&D? I guess we'll probably never know, but I wouldn't take as a given that inference is operating at a loss.
I found this: https://semianalysis.com/2023/02/09/the-inference-cost-of-se...
which estimates that it costs $250M/year to operate ChatGPT. If even remotely true $10B in revenue on $250M of COGS would be a great business.
As you say, we will never know, but this article[0] claims:
> The cost of the compute to train models alone ($3 billion) obliterates the entirety of its subscription revenue, and the compute from running models ($2 billion) takes the rest, and then some. It doesn’t just cost more to run OpenAI than it makes — it costs the company a billion dollars more than the entirety of its revenue to run the software it sells before any other costs.
[0] https://www.lesswrong.com/posts/CCQsQnCMWhJcCFY9x/openai-los...
CapEx vs. OpEx.
If they stop training today what happens? Does training always have to be at these same levels or will it level off? Is training fixed? IE, you can add 10x the subs and training costs stay static.
IMO, there is a great business in there, but the market will likely shrink to ~2 players. ChatGPT has a huge lead and is already Kleenex/Google of the LLMs. I think the battle is really for second place and that is likely dictated by who runs out of runway first. I would say that Google has the inside track, but they are so bad at product they may fumble. Makes me wonder sometimes how Google ever became a product and verb.
That paragraph is quite clear.
OpEx is larger than revenue. CapEx is also larger than the total revenue on the lifetime of a model.
Obviously you don't need to train new models to operate existing ones.
I think I trust the semianalysis estimate ($250M) more than this estimate ($2B), but who knows? I do see my revenue estimate was for this year, though. However, $4B revenue on $250M COGS...is still staggeringly good. No wonder amazon, google, and Microsoft are tripping over themselves to offer these models for a fee.
You need to train new models to advance the knowledge cutoff. You don't necessarily need to R&D new architectures, and maybe you can infuse a model with new knowledge without completely training from scratch, but if you do nothing the model will become obsolete.
Also the semianalysis estimate is from Feb 2023, which is before the release of gpt4, and it assumes 13 million DAU. ChatGPT has 800 million WAU, so that's somewhere between 115 million and 800 million DAU. E.g. if we prorate the cogs estimate for 200 DAU, then that's 15x higher or $3.75B.
1 reply →
> Obviously you don't need to train new models to operate existing ones.
For a few months, maybe. Then they become obsolete and, in some cases like coding, useless.
But assuming no new models are trained, this competitive effect drives down the profit margin on the current SOTA models to zero.
1 reply →