Comment by Jensson
6 hours ago
Humans learn to be a next token predictor as a kid when they learn to speak, an LLM cannot learn to be a next token predictor or anything of the sort, we have no clue how you could have an LLM learn human language just based on a thousands conversations with a human.
You don't see how that is very different? For an LLM to be as smart as a human it has to be able to learn like a human. Like you don't evaluate how smart a human is based on how much he knows, you evaluate it based on how fast he learns. And LLM are so bad at learning its ridiculous, they lack that part of the brain that lets humans be smart and learn so fast and easily.
> For an LLM to be as smart as a human it has to be able to learn like a human.
"For a plane to fly as well as a bird it has to be able to flap its wings".
"For a submarine to swim as well as a fish it has to be as light as fish".
> "For a plane to fly as well as a bird it has to be able to flap its wings".
> "For a submarine to swim as well as a fish it has to be as light as fish".
These are false equivalences. The post you're responding to defined intelligence as learning rate. LLMs unequivocally do not learn. You can disagree with OP or agree, but what you have done is out of bounds. You're implicitly claiming that LLMs learn, albeit differently from humans. This is categorically false, unless you count training as some kind of "learning".
That's of course absurd, almost no user of an LLM also trains it. Instead they rely on queries submitted to pre-trained LLMs ("inference"). And don't bother yapping about context windows, it's just not anything like learning.