Comment by r_lee
2 days ago
depends on if they're cost efficient. if there comes some kind of "good enough" inference chips and so on (which i believe will happen), then those will simply provide the most tokens per dollar and make open models even cheaper
Yeah; the open model markup goes to zero. The manufacturers of the "good enough" inference chips are probably going to be some subset of NVidia, AMD, Apple, smartphone SoC manufacturers, and (long shot) Intel.
I shortened my list to NVidia and AMD because putting Apple in muddies my point: I'd argue they'll be able to sell laptops + phones in that world, but only if you can run frontier open models on their devices, and do so efficiently. This is definitely not the case for their in-house AI offerings, like Siri, but it is already true if you install llama.cpp or whatever. I suspect that AI will not be a big revenue source for them (unless they completely turn around their software engineering org), but instead will be a "good enough to keep people from dumping their iCloud subscription" offering instead. Siri is basically that: It's completely useless, but people aren't switching to android because of it.