Comment by dyzone
3 hours ago
It tells me that they have some kind of insider knowledge that the models have hit their limits and won't be getting much better, and it makes sense economically speaking to just bake the current models and use them for the next 5-10 years. Looks like we're near the top of the S curve.
I’m not sure we are at the top. Swarmed lower tier models appear to scale up in competence. So something like a Fable swarm running at 100x the speed might provide surprises.
But ”top of the curve reached” feels like the likelier scenario.
A model you can run for practically no cost is a new proposition. It is the CPUification of AI. Sure there be supercomputers but you PC will be pretty super too.
What would possibly tell you that?
Expensive as fuck to make chips, only makes sense if you believe whatever model you're creating a chip out of will not become completely irrelevant in 5-10 years.
There’s lots of use cases where the current models do fine though. A chip that can run a current cheap model at 100x would be amazing for things like detecting prohibited content on Facebook. You don’t need the 2030 equivalent of Fable for that, you need something that can cheaply process insane numbers of posts per day.