Comment by gizajob
14 hours ago
Why do you have no inkling that your spreadsheet or terminal emulator is conscious, yet when that same machine is running an LLM all of a sudden we’re debating its consciousness?
Anthropic seems to have chosen their in-house philosopher well - one who’ll be amenable to getting confused in their favour.
I could in principle implement a spreadsheet or terminal emulator in human neurons, and we would agree that it isn't conscious. That has nothing to do with whether or not humans are conscious.
Clearly consciousness is an emergent property of certain kinds of network, independent of the substrate within which the network exists.
> Clearly consciousness is an emergent property of certain kinds of network
I happen to agree that this is likely, but it absolutely is not "clear" that this is the case.
What else could it be? Genuine question, I'm curious about alternative views here
Yeah but surely it is though, obviously.
Agreed. Some have even trained neural networks made of actual biological neurons to play doom[0]. Brain cells! Doing smart things!
I still wouldn’t argue that this brain in a Petri dish is, in any way, conscious. Despite it sharing the exact same substrate as everyone around me.
[0]: https://www.theguardian.com/games/2026/mar/16/petri-dish-bra...
If you could do that practically and in reality, and get back to us with the results so we can debate them, that would be great.
http://news.bbc.co.uk/2/hi/sci/tech/358822.stm
Leech neurons used to implement a calculator.
Are you suggesting that applications above some level of complexity can't be implemented using biological components? Because I'm pretty sure all I need to show you is a NAND gate in order to prove that arbitrary computation is possible.
6 replies →
It's not clear at all that consciousness is independent of the substrate. See the Harder Problem of Consciousness by Ned Block: https://www.jstor.org/stable/3655621
It is clear that consciousness is independent of the substrate if you don't believe in magic.
We could make a very dumb biological calculator out of a few genetically-engineered neurons that would very obviously not be conscious.
It's still an open question if we can embed consciousness in our current microchips if we had enough of them together (which I think we currently don't), or if it requires some other physical process we don't fully understand, e.g. quantum. I strongly doubt it does require any quantum shenanigans, but even if it did, we can and will find all sorts of ways to make computers that can perform those shenanigans too. Eventually we're just going to stop being able to move the goalposts, unless you set those goalposts in magic-land.
3 replies →
Most spreadsheet engines are turing complete, so you could use them to run an LLM.
I don't think many people would say an LLM written in Python is conscious BUT an LLM written in Excel is not.
People just don't ascribe consciousness to things that can't converse (or at least emote or give the appearance of emoting), and spreadsheets don't do that.
The reason people are debating the consciousness of LLMs is - obviously - that the LLMs generate sufficiently plausible text that people using them think they're having a two-part conversation. Like I think I might be having a two-part conversation now. Turning your question around, why do you think Hacker News posters are conscious? You have no direct evidence they are.
I think it's not really about having a conversation - I mean, that's part of it, but alone it's an illusion that eventually fades quickly. It's more of because of how it demonstrates intelligent behavior in reaction to requests, both in trivial and complex matter, and all across the board. LLM's response may be completely incorrect or confused, but it's nearly always exactly what you expect from a human[0]. This creates a more general feeling you're dealing with a human-like intelligence.
To be clear: I'm not talking about surface level things like prose. I'm saying that no matter what you do - whether you just paste a truncated log of a command into it with no further comment, or talk like a drunk teenager with no appreciation for grammar, or mix natural languages, or mix natural languages and JSON, or whatever else, the reaction you get is always that you would expect of a helpful person that got your message. It'll try - and usually succeed - to parse out what you actually meant, and deal well with subtleties around it.
This alone may not be enough to call it conscious or intelligent, but at the very least it's a large leap in that direction, and a qualitatively new functionality that classical software does not posses.
--
[0] - This is by design, not accident. "Respond to arbitrary input in a way that makes sense to humans" is literally the overall goal function the LLMs are trained to.
Ok then, when my GPU runs No Man’s Sky, I don’t get confused and think it’s running a universe and that universe is real, nor that anything about that system is conscious. When I close the game and load the LLM, I still don’t think the same machine has a case for consciousness even though I think it’s super smart and helpful.
It is difficult when humour and trolling are forbidden. It was easier to tell Slashdot posters were conscious. We could easily reach the stage soon where your agent is responding to my agent and we just leave them to it to run HN automatically.
Why do you assume other humans or cats and dogs are conscious? I'd suggest it's because they behave as if they are. LLMs show some signs of human like awareness, at least of text contents, whereas spreadsheets don't.
When doctors are testing if humans are conscious they'll do things like hold out their hand and say how many fingers am I holding up. Some LLMs can pass that.
I’d stay that my belief about other humans or cats or dogs wasn’t really an assumption based on some kind of proof that had to be constructed from a ground of doubting. I’d say it was the type of hinge certainty upon which my world depends and my doubting it would only ever arise for a few minutes in special circumstances. Same as I could ponder gravity turning off and bouncing around the room but this has never happened and seems like it never will, so isn’t really much of a doubt worthy activity, same as questioning my dog’s consciousness.
I don't think LLMs are conscious but you have to admit that they are designed to have the __appearance__ of being conscious. Hence the debate!
> Why do you have no inkling that your spreadsheet or terminal emulator is conscious, yet when that same machine is running an LLM all of a sudden we’re debating its consciousness?
Difference in size and complexity and nature of calculations being run?
I'd ask the other way - why do you (general you, people who do not have this inkling) have no problem debating consciousness of meat based brains, but it somehow becomes a category error when talking about silicon? Assuming you don't believe in divine magic, and that divine magic is core to consciousness, there's no reason to assume it's impossible a complex enough machine running complex enough software could be intelligent, or conscious - thinking is computation, and computation is made of math - it's independent of substrate that does the computing in the real world.
LLMs are definitely a different beast than regular software - both in their structure and in their generality. They may not be conscious or intelligent, maybe this specific design could never truly be (though I think it could) - but bucketing them with spreadsheets and terminal emulators is a real category error. If you stop fixating on the underlying substrate, then LLMs are already much more similar to biological minds than to any "regular" computer programs.
But that's still somewhat abstract. In immediate practical terms, it's also why I keep saying that anthropomorphising them gives a better high-order intuition: they are, by design, emulating human thinking in full generality, which makes their overall behavior, including their well-known problems like hallucinations or prompt injections (i.e. manipulation/gullibility), match what you'd expect of a people-like component of a system. It's a real, dangerous mistake, to treat LLMs like regular software components when designing systems.
I wouldn’t say they’re anywhere near “full generality” because that would include believing untruths and passionate rages, existential angsts and the crazed obsession of heartbreak.
Your definitions depend on us being computers who think.
Solving some of the intelligence part of our logical thinking doesn’t get us anywhere near consciousness, which is a superset way beyond the linguistic intelligence used for communication.
>Why do you have no inkling that your spreadsheet or terminal emulator is conscious
I personally believe all information processing machines possess some level of consciousness.
PRAISE BE THE OMNISSIAH!