Comment by AntiUSAbah
1 day ago
We are throwing unheared amounts of money in AI and unseen compute. Progress is huge and fast and we barely started.
If this progress and focus and resources doesn't lead to AI despite us already seeing a system which was unimaginable 6 years ago, we will never see AGI.
And if you look at Boston Dynamics, Unitree and Generalist's progress on robotics, thats also CRAZY.
If I'm reading you right, your opinion is essentially: "If building bigger and bigger statistical next word predictors won't lead to artificial general intelligence, we will never see artificial general intelligence"
I don't know, maybe AGI is possible but there's more to intelligence than statistical next word prediction?
Its not a statistical next word predictor.
The 'predicting the next word' is the learning mechanism of the LLM which leads to a latent space which can encode higher level concepts.
Basically a LLM 'understands' that much as efficient as it has to be to be able to respond in a reasonable way.
A LLM doesn't predict german text or chinese language. It predicts the concept and than has a language layer outputting tokens.
And its not just LLMs which are progressing fast, voice synt and voice understanding jumped significantly, motion detection, skeletion movement, virtual world generation (see nvidias way of generating virutal worlds for their car training), protein folding etc.
I'm sorry but the input to a model is a sequence of tokens and the output is a probability distribution of what's the most likely next token. It's a very very very fancy next token predictor but that is fundamentally what it is. I'm making the argument that this paradigm might not give rise to a general intelligence no matter how much you scale it.
31 replies →
> Its not a statistical next word predictor.
it absolutely is a next word predictor
LLM proponents believe that these higher level encodings in latent space do in fact match the real world concepts described by our language(s).
However, a much simpler explanation for what we see with LLMs is that instead the higher level encodings in latent space match only the patterns of our language(s), and no deeper encoding/understanding is present.
It's Plato's Cave - the shadows on the wall are all an LLM ever sees, and somehow it is expected to derive the real reality behind them.
1 reply →
> And if you look at Boston Dynamics, Unitree and Generalist's progress on robotics
Their progress is almost nought. Humanoids are stupid creations that are not good at anything in the real world. I'll give it to the machine dogs, at least they can reach corners we cannot.
I found there demonstration at the CES this year very spectacular: https://www.youtube.com/watch?v=YIhzUnvi7Fw
I can also recommend looking at Generalist: https://www.youtube.com/@Generalist_AI
> Their progress is almost nought.
How can you say the advancements since Honda's asimo robot amount to "almost naought"?
Not sure if you're being sincere or sarcastic but some of us have lived through several AI winters now. And the fact that such a phenomenon exists is because of this terrible amount of hype the topic gets whenever any progress is made.
Which ones? At least in the last 4 years, there was no AI winter.
The late 70s, again in the late 80s. See wikipedia.
https://en.wikipedia.org/wiki/AI_winter
1 reply →
History started well before 4 years ago
3 replies →
> Progress is huge and fast
is it? we're currently scaled on data input and LLMs in general, the only thing making them advance at all right now is adding processing power
Same thing happened with self-driving cars. Oh and cryptocurrencies.
Self-driving had never the amount of compute, research adoption and money than what the current overall AI has. Its not comparable.
Crypto was flawed from the beginning and lots of people didn't understood it properly. Not even that a blockchain can't secure a transaction from something outside of a blockchain.
The LLMs are flawed, and lots of people don't understand them properly.
1 reply →
> Self-driving had never the amount of compute, research adoption and money than what the current overall AI has. Its not comparable.
$100+ billion in R&D and it's not comparable... hmm
> Self-driving had never the amount of compute, research adoption and money than what the current overall AI has.
And yet they don't do really good jobs with pretty much anything, save for software development, to which people still seem pretty split as far as it being a helpful thing. That's before we even factor in the cost.
6 replies →