Comment by bitwize
2 months ago
Indeed. Local models becoming available and halfway decent don't obviate the laws of scale. And because there's no ceiling to what scaling more will buy you in terms of capability, there's no reason not to scale more, there's no incentive for billionaires not to grab all the fab capacity they can.
Enjoy paying $1000 or more for a little 4 GiB cloud terminal that connects you to all your online accounts where all your actual work gets done. This is the future.
>there's no ceiling to what scaling more will buy you in terms of capability
This is highly doubtful.
Rule of thumb: everything people think is exponential is actually an S curve.
There's a limit that won't be breached without a fundamental breakthrough in physics of computation, but we're not there yet by a long shot. You can train bigger models, faster, and infer with them faster and more precisely, by throwing more compute at the problem for the foreseeable.
At some point, and I can already see it, they’ll be better than us at writing code. We are still in the loop to coerce them into architecting well, but that’s nothing magical.
What’s frontier now is prosumer in a couple years and commonplace in a couple more.
better rule: exponentials are overlapping S-curves