Comment by baxtr
3 months ago
Isn’t that a bit like saying: storage is commodity and thus profit margins will be/should be low.
All major cloud providers have high profit margins in the range of 30-40%.
3 months ago
Isn’t that a bit like saying: storage is commodity and thus profit margins will be/should be low.
All major cloud providers have high profit margins in the range of 30-40%.
Storage doesn't require the same capex/upfront investment to get that margin.
How much does it cost to train a cutting edge LLM? Those costs need to be factored into the margin from inferencing.
Buying hard drives and slotting them in also has capex associated with it, but far less in total, I'd guess.
They don't, though! I can buy hardware off of the shelf, host open source models on it, and then charge for inference:
https://parasail.io, https://www.baseten.co
Yes, which is why the companies that develop the models aren't cost viable. (Google and others who can subsidize it at a loss obviously are excepted)
Where is the return on the model development costs if anybody can host a roughly equivalent model for the same price and completely bypass the model development cost?
Your point is inline with the entire bear thesis on these companies.
For any use cases which are analytical/backend oriented, and don't scale 1:1 with number of users (of which there are a lot), you can already run a close to cutting edge model on a few thousand dollars of hardware. I do this at home already
4 replies →
Yes you’re right. Capex spend is definitely higher.
In the end it comes all down to the value provided as you see in the storage example.
RAM also apparently.
this is slightly more nuanced, since the AI portion is not making money. it's their side hustle