> There's definitely going to be cheap or open source models
What makes you think your "cheap or open source model" running on your piddling desktop cluster will be able to complete against a SOTA one running in a billion-dollar datacenter?
It's a cyberpunk fantasy. It won't work out that way.
Local models that run on a laptop (not even needing a "cluster") are already better than ChatGPT from a couple of years ago. Yes, Claude and ChatGPT today are certainly better than these local models, but they can't keep getting better indefinitely -- there's only so much info to scrape. When they hit a plateau, it is only a matter of time that consumer hardware will catch up to it.
Maybe? We dont really know this right? People have been saying this for 5 years now and the models are still getting better. The companies running the frontier models have already scraped everything on the web, but the models are still getting better, even if it's only marginally better, with each release. Maybe eventually some company will actually achieve AGI/ASI, who knows..
I think the parent is speculating that there may be an order of magnitude improvement in the cheap / OSS model space such that one running on a piddling desktop cluster could match or exceed the capabilities of the current SOTA on billion-dollar datacenter.
> There's definitely going to be cheap or open source models
What makes you think your "cheap or open source model" running on your piddling desktop cluster will be able to complete against a SOTA one running in a billion-dollar datacenter?
It's a cyberpunk fantasy. It won't work out that way.
Local models that run on a laptop (not even needing a "cluster") are already better than ChatGPT from a couple of years ago. Yes, Claude and ChatGPT today are certainly better than these local models, but they can't keep getting better indefinitely -- there's only so much info to scrape. When they hit a plateau, it is only a matter of time that consumer hardware will catch up to it.
> but they can't keep getting better indefinitely
Maybe? We dont really know this right? People have been saying this for 5 years now and the models are still getting better. The companies running the frontier models have already scraped everything on the web, but the models are still getting better, even if it's only marginally better, with each release. Maybe eventually some company will actually achieve AGI/ASI, who knows..
I think the parent is speculating that there may be an order of magnitude improvement in the cheap / OSS model space such that one running on a piddling desktop cluster could match or exceed the capabilities of the current SOTA on billion-dollar datacenter.