Comment by jaggederest
14 hours ago
I guess I fundamentally disagree, because on general knowledge they beat any human, and on processing speed they beat any human, and on problem solving likewise. I just don't see a domain besides "retain change over time" that they fail on, and that is something you can do in the system rather than the model - coding agents can and do build their own memory system and use it, like an amnestic human writing notes.
I think what I'm objecting to is the idea that if it can't do every task that every human can do better than any human, it's not AGI - I think we have to allow that this is a fundamentally different form of cognition, so requiring a 100% match feels uncharitable. Every human has cognitive gaps that AI doesn't have.
I'm not sure there is anything to disagree on here - just different definitions of what we personally associate the term "AGI" with. There is no magic to the name, and however you choose to define it there will be less and more capable levels of AI that precede and follow it.
Separate from what you choose to call any given set of AI capabilities, learning is generally more than retaining change over time. Learning isn't the same as memorization - it's the difference between memorizing a stack of how-to books and going out and practicing the skill for yourself, and all that entails.
A learnt skill needs to be internalized such that in the future it's what is driving behavior, which means it needs to be in the weights. Imagine a robot trying to learn to play tennis where rather than updating it's connections between perception and action it's trying to store everything as a bunch of notes!
It's not just physical tasks where this applies - the same would be true to trying to teach a multimodal model to recognize different types of mushrooms from photos, or even learning a new language. If a system can truly learn for itself then it shouldn't need to be pre-trained with language, but rather should be able to learn by immersion and practice the same way we do, and the same way a dog learns to follow commands. You can't do this by leaving notes to yourself - the learning mechanism needs to be animal-like and update the systems weights in response to prediction failure - a runtime version of the way SGD-based pre-training updates weights in response to prediction errors.
> If a system can truly learn for itself then it shouldn't need to be pre-trained with language, but rather should be able to learn by immersion and practice the same way we do, and the same way a dog learns to follow commands. You can't do this by leaving notes to yourself - the learning mechanism needs to be animal-like and update the systems weights in response to prediction failure - a runtime version of the way SGD-based pre-training updates weights in response to prediction errors
I am not sure why any of that is necessarily true. I think you are crediting the human brain with a lot more than it actually does - memory is something in the direction of a fine tune on existing neural hardware, it's really not anything special, and I absolutely think that "just taking notes", when taken to a sufficient fidelity, is enough to pass that bar.
You can't take notes before you've learned language, so if that's how you propose to learn then you are doomed to build something with less learning capability than an human brain.
Maybe you don't care - if you just want to push to see how far an LLM can go that's fine. but it's not going to be superhuman-level, or even human-level, if it doesn't have the learning ability of a human. If it relies on language and notes to itself then it is just an LLM.
1 reply →