Comment by JCTheDenthog
11 hours ago
I mean, outside of our own internal experience of qualia, there's no way for us to prove anyone or anything is conscious. But my biggest issue with the debates about consciousness and LLMs is that the "LLMs are conscious" crowd usually has to presuppose a completely materialistic/mechanistic understanding of the universe and of our own consciousness, and that's something that we really don't know for certain. Also, if we were running LLMs on a mechanical or pneumatic or water based computer (albeit very slowly) then the non-electronic computer would be just as "conscious" as the normal electronic computer, and I think it would give a lot of the "LLMs are conscious" crowd pause to think that a bunch of pipes and valves can be conscious. I think there's a lot of magical thinking that arises from the fact that computers are electronic.
> a bunch of pipes and valves can be conscious
Billions * billions of pipes and valves can result in emergent behavior that appears conscious while at the same time the sound of a single independent water pipe can moan and sound like human speech or otherwise lifelike and evoke human emotions.
I think LLMs are doing both of these things and often people are more impressed by the independent fixtures (the moan) rather than the emergent behavior. Both the sound and the emergent behavior can be built on purpose or on accident.
I think it helps to look at this through an Information Theory lens. What information is coming into the system (the human or the machine)? What information goes out of the system which is novel? How much of this can be attributed to attempting to parse random noise aka. `Random_Imagination_Engine` vs something else? The number of inventors who come up with a breakthrough idea after mis-hearing someone is surprisingly high.
If we make the distinction between phenomenal consciousness from access consciousness we can see that LLMs clearly can make decisions based on input (A-Consciousness) but they probably don't have raw feelings and sensations (P-Consciousness).
> What information goes out of the system which is novel?
I think there is very little truly novel information. Most information including the information of "breakthrough idea after mis-hearing someone" is just a mix of previous information.
I guess you are already aware of that... just for completness sake.
You will need immense evidence to go against the entire arc of history where all bets about supernatural or aphysical elements have failed. Until you do so, the brain is another not fully solved physical science problem.
And yes, turing equivalence is turing equivalence, I don't see why a system of pipes can't make an AI.
We only need to assume the hypothesis that brains don't exceed the Turing computable to say that LLMs have the same computational power as a brain.
We can't prove it, but given the total absence of evidence of anything exceeding the Turing computable, as a hypothesis it is a reasonable one that would require truly extraordinary evidence to rise above "magical thinking".
Now, that is also far from proving they are "conscious".