Comment by atleastoptimal
1 day ago
This makes sense. However there is an issue where many people conflate "consciousness" with the ability to make novel insights, think genuinely, etc. They use this to claim that since AI is not conscious, AI could never actually "think" and is instead just always a regurgitation of its training data.
It is a natural human hubris to make our abilities seem unquantifiable and mysterious, but all the useful things the human brain does are just finding patterns in data, running lossy simulations, and estimating on abstraction, all things it is theoretically possible for any non-conscious machine to do.
Is there even an agreed and actionable definition of consciousness? I'm worried that if such a thing existed some humans would fail to measure up.
To the best of my knowledge, there is not an agreed and actionable definition of consciousness, and any attempt to make one comfortably fails to cleanly divide humans from machines.
It's more of a vibey term, and as such it is genuinely very difficult (perhaps impossible even) to concretely determine whether an LLM possesses consciousness. LLMs successfully express a lot of consciousness-like traits.
At some point you have to ask the question: does it even matter? If an LLM can sufficiently mimic consciousness, isn't that sufficient for us to treat it as conscious, even if it is in-fact not conscious (especially because we don't actually know)?
As I've mentioned somewhere else already and you pointed out; the issue is that consciousness is a loose term and and it's a semantical issue that should be resolved first. It won't be, because the murkiness is beneficial to the corpus, amongst other things. And to your point about the appearance of "consciousness" being enough imho Chiang explains fairly well why it's not.
There is a difference between saying software can never be conscious and saying the software we have today isn't conscious.
Yes. There are really three separate questions:
I'd assign probabilities of around 0.1, 0.2, and 0.9. My completely ignorant take is that we probably need something more "dynamic" than a bunch of transformer layers in order to produce consciousness, but I wouldn't be shocked to be mistaken.
I assign probabilities of zero to all 3. Computer program being conscious leads to ridiculous and obviously false conclusions (think about a person running a program using pen and paper for memory).
10 replies →
I think 10-20% chance is wildly generous. What specific mechanism makes you think there's a 10% chance that current LLMs are conscious?
4 replies →
I have yet to be convinced that LLMs can produce definitive knowledge that is not a result of combining previous information. Humans can (if they can't then science basically collapses epistemelogically, see: philosophical skepticism), but I see no evidence of LLMs doing it. And from the number of truly new ideas and concepts delivered by LLMs (exactly zero), I think it's reasonable to just treat them as induction machines for now, but to treat anything they "know" as a Gettier case.
I would like to push back on the idea that humans can provide definitive knowledge that is not a result of combining previous information.
Pretty much every single idea in science can be traced back to some combination of earlier ideas, and as they get earlier / simpler, they can be related back to some combination of direct observations.
It's not clear to me at all that our entire body of scientific knowledge can't be simply recreated by "combining results of observations + previous information". And LLMs can perform observations in addition to combine previous information, which in my estimate is genuinely sufficient for them to plausibly be able to rebuild all of science.
How can you say for sure that every effect must have a cause? How can you be any more sure about that there can't be an effect without a cause than the old believe that black swans couldn't exist that was so universally believed that it became a cliche ("rara avis in terris nigroque simillima cygno")?
If every piece of knowledge is created by combining previous ones, then no true grounded knowledge is possible. Every bit of knowledge is grounded by another bit of knowledge which is itself grounded on, ..., etc, etc.
https://plato.stanford.edu/entries/induction-problem/
1 reply →
> people conflate "consciousness" with the ability to make novel insights, think genuinely, etc.
The funniest thing is that LLMs will lap people in those capacities way before people who think like that accept that they might be conscious.