Comment by bloppe
2 days ago
Switching a complex cloud deployment from AWS to GCP might take a dedicated team of engineers several months. Switching between models can be done by a single person in an afternoon (often just 5 minutes). That's what we're talking about.
That means that none of these products can ever have a high profit margin. They have to keep margins razor thin at best (deeply negative at present) to stay relevant. In order to achieve the kinds of margins that real moats provide, these labs need major research breakthroughs. And we haven't had any of those since Attention is All You Need.
" Switching between models can be done by a single person in an afternoon (often just 5 minutes). That's what we're talking about."
Good gosh, no, for comprehensive systems it's considerably more complicated than that. There's a lot of bespoke tuning, caching works completely differently etc..
"That means that none of these products can ever have a high profit margin."
No, it doesn't. Most cloud providers operate on a 'basis' of commodity (linux, storage, networking) with proprietary elements, similar to LLMs.
There doesn't need to be any 'breakthroughs' to find broad use cases.
The issue right now is the enormous underlying cost of training and inference - that's the qualifying characteristic that makes this landscape different.