Comment by Deegy
4 hours ago
It is unclear. Everyday I seem to read contradictory headlines about whether or not inference is profitable.
If inference has significant profitability and you're the only game in town, you could do really well.
But without regulation, as a commodity, the margin on inference approaches zero.
None of this even speaks to recouping the R&D costs it takes to stay competitive. If they're not able to pull up the ladder, these frontier model companies could have a really bad time.
Probably it's "operationally profitable" when ignoring capex, depreciation, dilution and other required expenses to stay current.
Of course that means it's unprofitable in practice/GAAP terms.
You'd have to have a pretty big margin on inference to make up for the model development costs alone.
A 30% margin on inference for a GPU that will last ~7 years will not cut it