Comment by Tuna-Fish
20 hours ago
Either frontier AI will continue to get better steadily, until it's better than people at basically everything, and very weird things start to happen, or, it plateaus and the open weights models will eventually catch up.
There's a third possibility, which to me is also the most plausible. Models will keep getting better and better within certain domains, while never catching up with humans when it comes to other type of tasks. Like that elusive quality-without-a-name some have called "taste". Or knowing what UX is most intuitive to humans. Really, anything where we have to tap into our common humanity and our human experience in order to be of help to other humans and generally to life on earth. Which is an all-important skill, since the flowering of life on the earth is (or should be) the whole point of anything we do, after all.
We've spent a couple of millennia documenting that elusive thing called 'taste' in word, sound, image, lately in moving images. I'm pretty sure that even if you didn't have taste you could make a good simulation based on all that input.
I don't know, that sounds like a question of taste. :)
It can remix things people have already documented but can it synthesize an entirely new thing?
1 reply →
So what you're saying is that since there are so many examples of what humans have preferred in the past for these models to be trained on, surely there will come a time when they will know even better than us what we want and what we like.
1 reply →
Everyone knows the path followed by every single technology mankind has ever developed - including the incredible run that was Moore's law. Sure, some will think "This time is different" - but is it?
It is, because just like you mentioned, there's no exponential physics to back up growth. LLMs have to stop scaling up compute, because there's only so much resources.
But even if we think back to the previous decade or two (sans AI), most of tech has been a a red queen's race - developing and improving stuff, without any real substantial results.
Microservices and orchestration are not better than old monoliths. Electron apps are not better than native. Most programs people actually use have decades-old codebases.
Video games of today don't look substantially better or play different than ones from 15 years ago.
It doesn't need to run indefinitely. I don't think anyone sane thinks it's going to run indefinitely. It just needs to run long enough. Moore's law was what, 50 years? If AI keeps improving at the rate it currently is for 20 years, the world will be very different when that curve flattens out, in a way that makes most of our current concerns are entirely irrelevant.
Those cycles have shortened considerably. Just look at the lifespan of media, integrated circuits, software, products. There have been so many big model releases just in the last three months I've lost count. We're now closer to weeks between steps.
Maybe if China stops releasing capable models the pace will slow down a little, but they have lots of incentives not to.