Comment by jaggederest
4 hours ago
Okay, nowhere did I say it would be before they learned language. I said we could use models that can trivially build notetaking systems to auto-improve themselves by taking, refining, and updating notes as a memory system to exceed the x token context threshold.
> but it's not going to be superhuman-level, or even human-level, if it doesn't have the learning ability of a human. If it relies on language and notes to itself then it is just an LLM.
Why do you think that? You do a lot of assuming a thing to be true without any clear reasoning or evidence. I'm talking about a system, including a LLM, that acts in a way that is equivalent to or better than an average human at a large range of tasks, such that is generally capable of learning and problem solving similar to the degree of success humans have. All these specific criteria about what is "actually human" is not the point. We know how to make more humans, I'm talking about a form of intelligence that we are creating that is genuinely alien, so holding to "well it doesn't work the same as a human" is an impossible bar. If you reverse the situation, humans are by no means LLM-equivalent intelligences - perfect memory of long form content, rapid calculation, instantaneous tool calling - a hypothetical silicon based lifeform would dismiss us as "really not intelligent at all, they can't even do multiple floating point calculations per second in their head", and I think that would be similarly silly.
OK, so we're basically talking about different things. I'm talking about building something that has animal/human intelligence and learning capability, and would be able to first learn language for itself (no pre-trainiing), and then proceed from there.
You're talking about building an "alien" intelligence, apparently something based on an LLM, where language is baked in from the start and can therefore be used as the basis of some type of learning (or at least memorization).
Sure an LLM can be super-human at specifc things like math where the traditional computer strengths of compute, memory, etc apply, but in the context of AGI (generality, not a bag of narrow intelligences) you can't call something super-human if it can't at least also do the majority of things that a human can do, and not having learning ability as powerful as a human seems to me to be a pretty massive omission.
If WE didn't have ability to learn language, then we'd not be here talking about it, and there would be no LLMs. Having a fundamental ability to learn, isn't just useful to learn language from scratch, or for learning all the non-linguistic skills a human is capable of, but would also support an AI that can learn things that we cannot (e.g. could give it additional "senses" like global pressure/temperature inputs so it could "see" the weather and learn global patterns).