Comment by briian
2 months ago
This reminded me that the economies of scale in AI, especially inference, is huge.
When people say LLMs will be commoditised, I am not sure that means that the market is going to be super competitive. As the economies of scale of AI get even bigger (larger training costs + batch inference etc.) it just seems likely only around 3 companies will dominate LLMs.
For inference cost, I don't see how this is different from cloud providers vs dedicated server providers, where AWS is 5-10x more expensive than hetzner.
Somehow cloud providers manage to add lots of extra-cost on offering.