Comment by stillpointlab
1 day ago
I mean, if I imagine Anthropic giving away unlimited Sonnet 4.5 away at $20, I would still be paying the $200 for fable.
It is a bit like saying "why would you hire someone with a doctorate when you could get unlimited high school grads". How appealing that sounds depends on your needs.
Right, and while there are needs that require a doctorate, having unlimited high school grads would be immensely useful for many many tasks.
The ability levels of the cheap models are encroaching on the abilities of the frontier models faster than frontier models are expanding their abilities. If we haven't already, we will very soon reach a "good enough" state where having the "best" model matters less and less and less.
By analogy, if you buy a new computer, do you get the absolute fastest CPU available? Maybe, depending on your workload. But if you're 90% of the population, you get the cheapest one that has enough power to meet your expected workload, which is mid-range, not top of the line.
I think there is growing confusion in this discussion. I am saying "I see the fixed nature of ASICs creating a barrier to their adoption despite how cheap they are".
Many of the responses seem to say "but there is a market for cheap models".
There is a world where we have cheap models and we don't have ASICs powering them. Things like TPUs and NPUs, which are programmable, are likely to fill that role. They are optimized for inference while also allowing different (and updated) models to run on them.
Given two companies competing on the cheap end. First company goes TPU, second goes ASIC: who wins? My bet is on TPU since they can update their model, even if their hardware is slightly more expensive and slightly slower, since the optionality of new models beats the performance gap. That may not hold forever but given the pace and volatility of the current LLM market, I believe it will hold for some time.
The reason you are getting "but there is a market for cheap models" as a response, is people saying in a roundabout way "but soon models may not need to be updated".
Models like Kimi 3, GLM 5.2, or even Fable 5 for that matter are reasonable to burn to.ASIC because they are over the threshold of "good enough to be generally useful", something that will continue to be true in the future.
Most people do not need the latest model, they need a sufficient model. If I had Fable 5 on an ASIC, I imagine I would use that and ignore paying API rates for Fable 6.
3 replies →
How much compute does the cheap model require to operate? Power + hardware amortized over a couple years.
It’s like how most people on here would want a loaded MacBook Pro or RTX 5090, but chromebooks and iGPUs do volume.
There’s absolutely a place for a lifestyle subscription to a sonnet model that you could just use everywhere all the time.
Yes, but the context of the discussion is "who wins the war and how". And you are moving the goal posts very far from the original claim, which is that the speed with which companies can get their models onto ASICs will be the determining factor.
Sure, but you’re undoubtedly the tip of the spear. Lots of people don’t need that.
What percentage of white collar work these days requires doctorate level thinking all the time?