That's a good point. If we don't figure out how to design for what we call consciousness it might be that what emerges from some future neural network is an alien mind that's very different from what humans would call conscious. Could that be called AGI?
That's still very distant from what people are calling AI today.
> very different from what humans would call conscious
I mean, what would you call conscious? The word literally means “aware; responding to one’s surroundings.” By that definition most any animal is conscious and LLM+Harness combos have been conscious for a while.
I think the real issue is that when most people refer to consciousness, they have their own subjective experience in mind which strongly resists any tidy definition. I think it’s extraordinarily unlikely LLMs have anything like this, but they are far more able to effectively respond to their surroundings than most animals and in some areas better than humans.
So if you’re waiting for proof that an LLM has an inner life basically equivalent to your own, you’ll be waiting a long time. After all, other humans can’t even prove the fact of their own consciousness to you! They could just be replaying their training data at you in a way that is merely a convincing but false simulation of the true consciousness which you experience inside your head.
I strongly disagree that llm's are conscious of their environment. Even an insect reacts to light and someone attempting to swat at it. An llm barely even receives input from its environment.
people define consciousness quite differently but it generally has to do with phenomenal experience. your provided definition would make a self-driving car conscious, which is fine to argue, but probably not intended.
Nope, animals are conscious and yet not AGI, so the two aren't equivalent.
Could consciousness emerge from any system capable of AGI? I doubt it: intelligence is only one axis, and consciousness probably depends on others, like memory, self-reflection (one's output feeding back as input), and continuous operation that reacts to events from both the environment and the self.
I’d argue intelligence is closer to being able to survive and fend for oneself in a dynamic environment than it is making the next scientific breakthrough.
Yeah mind boggling for many here I’m sure.
That’s why the bizarre paradox is llm’s will be better than humans at some complex things but useless at many things that humans regard as being simple. E.g the leap of faith re. LLM’s and robotics.
That's a good point. If we don't figure out how to design for what we call consciousness it might be that what emerges from some future neural network is an alien mind that's very different from what humans would call conscious. Could that be called AGI?
That's still very distant from what people are calling AI today.
I mean, what would you call conscious? The word literally means “aware; responding to one’s surroundings.” By that definition most any animal is conscious and LLM+Harness combos have been conscious for a while.
I think the real issue is that when most people refer to consciousness, they have their own subjective experience in mind which strongly resists any tidy definition. I think it’s extraordinarily unlikely LLMs have anything like this, but they are far more able to effectively respond to their surroundings than most animals and in some areas better than humans.
So if you’re waiting for proof that an LLM has an inner life basically equivalent to your own, you’ll be waiting a long time. After all, other humans can’t even prove the fact of their own consciousness to you! They could just be replaying their training data at you in a way that is merely a convincing but false simulation of the true consciousness which you experience inside your head.
I strongly disagree that llm's are conscious of their environment. Even an insect reacts to light and someone attempting to swat at it. An llm barely even receives input from its environment.
2 replies →
people define consciousness quite differently but it generally has to do with phenomenal experience. your provided definition would make a self-driving car conscious, which is fine to argue, but probably not intended.
The fact that something is alien doesn't mean that's not conscious, humans aren't the pinnacle of biological development/evolution.
Nope, animals are conscious and yet not AGI, so the two aren't equivalent. Could consciousness emerge from any system capable of AGI? I doubt it: intelligence is only one axis, and consciousness probably depends on others, like memory, self-reflection (one's output feeding back as input), and continuous operation that reacts to events from both the environment and the self.
The parent's question could better framed as "is consciousness a requirement for AGI?"
[dead]
Your definition of AGI is flawed.
I’d argue intelligence is closer to being able to survive and fend for oneself in a dynamic environment than it is making the next scientific breakthrough.
Yeah mind boggling for many here I’m sure.
That’s why the bizarre paradox is llm’s will be better than humans at some complex things but useless at many things that humans regard as being simple. E.g the leap of faith re. LLM’s and robotics.
[dead]