Comment by sholain

" Switching between models can be done by a single person in an afternoon (often just 5 minutes). That's what we're talking about."

Good gosh, no, for comprehensive systems it's considerably more complicated than that. There's a lot of bespoke tuning, caching works completely differently etc..

"That means that none of these products can ever have a high profit margin."

No, it doesn't. Most cloud providers operate on a 'basis' of commodity (linux, storage, networking) with proprietary elements, similar to LLMs.

There doesn't need to be any 'breakthroughs' to find broad use cases.

The issue right now is the enormous underlying cost of training and inference - that's the qualifying characteristic that makes this landscape different.