Comment by kami23
1 day ago
This read like poetry to me. Thank you for sharing it.
I have a linguistics background and a lot of my philosophizing lately has been on whether or not the emergent abilities of the LLMs is deep down a similar mechanism that creates our consciousness.
For a little bit I was working on having linguistics based evals for a kaggle competition. My challenge was whether or not I could mask things well enough to not trigger its internal state of certain phenomena, and that sent me down a rabbit hole that I'm still exploring.
This story resonated with a lot of questions that can come out of figuring a good solid answer to the what is consciousness question. The one I triggered for me is: Is our perception of time just a slow thread in the giant GPU we are running the universe on? Or more generally, what is time? That's a fun YouTube rabbit hole if you ever need one.
Regarding consciousness , I like the explanation by neuroscientist Ramachandran:
https://www.edge.org/3rd_culture/ramachandran07/ramachandran...
In short, as far as I can remember: evolutionary, it makes sense to understand other humans, to feel what they feel(empathy - the mirror neurons system), and simulate their thinking and feelings.
And once we have those systems, we can also use those on ourselves. And that's consciousness.
Edit:And I wonder if this is a testable hypothesis, in a simulation.
This way of thinking can only explain externally-visible parts of consciousness. It does nothing to address internal experience of being conscious and qualia. I don't think the internal experience has any bearing on physical reality (P-zombies would act the same externally) which makes this something outside of the realm of currently understood physics.
The internal experience has bearing on physical reality right here, because without it you wouldn’t have written about it and caused these words to appear on my screen in physical reality. It affects your thinking, and hence your actions and utterances in the physical word.
For reasons like that, I don’t think that P-zombies are possible.
6 replies →
Qualia are the inside view of sensory data and reward signals.
Think about it from an evolutionary perspective:
Animals that step into a lava flow or forest fire don't reproduce. Eventually some evolve the ability to detect intense heat from a distance, and pain as soon as tissue destruction is imminent. They do not have nor need a general understanding of the dangers of heat, or even conscious awareness that they've stepped on a hot coal.
The pain signal compels them, but that is very low level machinery. It had to continue compelling beings that developed larger and more sophisticated brains that are capable of abstract thought and reasoning. Feeling pain is one of the lowest level parts of the brain telling the higher parts exactly what its going to do in terms that permit no disagreement.
Internal to what? The brain is not a monolithic thing, it is different parts communicating and interconnecting. When the connection between the halves is cut, the person objectively becomes two people, but still experiences and presents themselves as one. Observing ourselves is just one part of the brain responding to another, or theorizing on past behavior. There is probably no actual introspection going on inside of the human brain, only the perception of one.
1 reply →
There are some evolutionary theories why did qualia("phenomenal awareness") evolved. I don't know much about those, so i can't write about that.
Oh, that's a very interesting hypothesis!
Not sure about taking it down to the level of consciousness, but makes sense regarding the sense of self, the conceptual experiencer, the perceived center of experience. It agrees well with the observation I have made again and again they my sense of self is much stronger when I'm around people, and stronger still when I'm in a context where I don't know people and/or am uncertain in social rules.
This can be as immediate as dancing in a club, and closing my eyes I feel open, free, still, the body just flowing, then opening my eyes and feeling the cage of categorization of the world, relating my self to other people as a major function, coming right back.
Also being alone in nature for me makes the sense of self drop. Without intention, spending even just a few hours alone in a forest seems to quiet down the part modeling my self in relation to the world so much. There's no need for it there. I'm not a person in a forest; I become the trees, the birds, the rustling of the leaves, the sun shining through the canopy.
I agree about the forest part. and your comment was interesting.
I know that the part of the brain responsible for the self thoughts is called the "default mode network". and meditation can reduce it's activity, i.e. the internal monologue stops, but also it can be measured via FMRI.
So i wondered: are the mirror neurons part of the "default mode network"? I asked claude that, he said no, they are two different systems.
So maybe the mirror neurons, those responsible for empathy, "to feel as someone else" are also responsible for becoming the trees, the birds and the rushing of the leaves?
It's interesting, but the explanation seems to exclude the possibility that my dog has some sense of self, so I find it suspect.
Don't dogs have similar social issues?
1 reply →
There's a really, really big gap there. It reminds me of South Park's underpants gnomes.
1. Simulate others' thinking and feelings.
2. ???
3. Consciousness!
Why would one lead to the other? How would one lead to the other?
This read like poetry to me
It's terrific, but the poetry is from the original it links to, in case you didn't realise.
It's a brilliant and timely update though.
Aside, there are various recorded versions including video on YouTube but this is my favourite, a radio play:
They're Made Out of Meat
https://www.wnycstudios.org/podcasts/studio/segments/168264-...
[dead]
For obvious survival reasons we evolved to have sensory/cognitive access to our own activity, self-monitoring, and self-modeling ourselves.
The self-modeling, is in such a tight loop, it melds "ourselves" and our model of ourselves, our thinking and choices, and experience of our thinking and choices, into one component.
Like you can't analyze half a wheel of a bicycle and be talking about the same thing.
This awareness, increased modeling, control, feedback loop has tightened up over many stages. Just a few:
1. The body-sense loop
2. The internalized-environment-model loop
3. The body-internal-function loop
4. The body-internal-model loop
5. The emotional-cognitive loop
6. And finally, the tightest loop of all, our high-level cognitive activity, experienced as feedback directly, our self-model, and our self-direction, all merged into one thing.
We literally spend almost all day, every day, thinking about ourselves, in terms of our inner self.
That is consciousness. Rich self awareness, a merger of self-model and self-direction, and all in service of understanding and managing ourselves. Hw we can leverage our greatest tool, our self-directable mind, its habits, views, and behavior.
This wasn't an accident. A happy side-effect of our brains. It is a biologically evolved focusing of our highest-level behavior, with tight feedback, constant self-modeling and continuous focus on our inner status as motivation and most privileged object of our control. It has been ruthlessly optimized for, for a very long time.
> We literally spend almost all day, every day, thinking about ourselves, in terms of our inner self.
> That is consciousness.
So thinking is consciousness?
Can there be consciousness without content? E.g. can I just be conscious of being conscious? If so, consciousness cannot be defined as the thing(s) we're conscious of.
> Can there be consciousness without content? E.g. can I just be conscious of being conscious?
Being conscious of being conscious means that there is content. You are conscious of something.
It’s a bit like a Gödel statement that quotes itself, that is a statement about itself. It doesn’t mean that it has no content.
Thinking isn’t consciousness. Consciousness doesn’t require thinking, it only requires perception. The perception of a process of perception within the same mind might constitute consciousness.
I think this is exactly it, but let me ask another question (which is not rhetorical, I really don't know). Does the fact that one can describe what consciousness is and where it came from in humans help them to detect it in non-human and/or non-biological entities?
That is a really good point. Yes, I think function is diagnosis on this.
Constant self-awareness, self-experience, self-focus, self-management, and self-improvement of one's own self (mind), is going to be an adaptive behavior for anything intelligent with resources to leverage. Whether truly independent, or highly motivated to serve others. The mind is the greatest tool.
I think that is more than simply a good functional definition of consciousness. How could all that integration and self-integration not be conscious.
Yeah, I currently suspect that consciousness is an emergent property. I read elsewhere (it's somewhere in my HN history, I'm sure) that the biggest compute we can currently muster is something like three or four magnitudes away from the number of neurons / connections (or their analog) that our brains have, so it may be a while until we can expect to see it in our machines. But, if the emergent phenomenon hypothesis is correct, then we eventually will. I'm more scared than pleased by the prospect, but there you are.
>consciousness is an emergent property
You would really like Michael Pollan's latest book [1], entirely devoted to his exploration of consciousness researchers' POVs on this exact topic.
My favorite quote is that ~"perhaps Descartes was only half-wrong when suggesting I think, therefore I am; it seems rather closer to I FEEL, therefore I am."~
[1] <https://www.amazon.com/World-Appears-Journey-into-Consciousn...>
----
I've grown thousands of plants; I've read two of the author's other books devoted to plants; in this book Pollan makes compelling arguments for plant sentience (over a much-longer timeframe).
Sure, perhaps plant consciousness is a bit of a stretch, but they're certainly intelligent and curious creatures. He makes both arguments supporting plant volition.
----
If you haven't seen My Octopus Teacher (Netflix), do. I'm a bald 275lb bluecollarguy... and I wept/awed (both). So beautiful, we bundled neurons.
----
Bonus quote ~"color is where reality and magic appear as-if together"~ [color isn't real, but is perceived]. We most-often see what's most-predictable, not necessarily what we actually detected [in the case of color: nothing but nanometers].
Just to be sure: The "neurons" in today's AI have nothing to do whatsoever with real neurons.
What we can do is simulate very simple brains by simulating relatively few neurons as they appear in worms. In this sense we are multiple magnitudes away where the increasing complexity implies exponential increasing difficulty.
I would think we are so far away that there will be unknown unknowns we encounter on the way.
Yes, physically absolutely nothing. But conceptually they seem to to form this very generic function from inputs to outputs that neurons also form.
5 replies →
> Just to be sure: The "neurons" in today's AI have nothing to do whatsoever with real neurons.
Yeah, but it's hard to explain this to people, especially AI-pro people. Too many are convinced that all we are doing is a cut-down version of the human brain, and it's hard to explain to them that, no actually, we aren't modeling the human brain to the level of granularity you think we are.
Personally I'm not a fan of the emergence story, for a number of reasons. First is, it doesn't really make sense for consciousness to have emerged gradually through natural selection. When we and animals are conscious, the whole brain is coordinated and works to for example turn off consciousness when we sleep. And as someone else mentioned, animals with much fewer numbers seem to have a similar consciousness to us.
If consciousness really evolved gradually, you would expect to see for example dogs or gorillas having less of it, but if they has less of it, why does it function the same way? Like for example animals can be scared, happy, anxious etc, they can experience the full range of emotions and thoughts, so their conscious experience seems just as rich as ours. What I mean by this is, if you can be "less conscious", then what does that mean _exactly_? Is it that you have less content in consciousness, or is it that you feel more like you are asleep? Or something else? We don't have any examples in animals of "less conscious", I would argue.
This makes me think that rather than having emerged gradually, evolution found a mechanism by which consciousness exists, and then some animals have that mechanism and others don't. I think that if it is a mechanism, then this mechanism is located in one part of the brain, not many parts functioning together (though one possibility is that this mechanism coordinates brain activity in such a way to enable consciousness).
I proposed in another comment that consciousness and self-awareness are at least close cousins, and perhaps the same phenomenon. If that's true, then that's an axis upon which you might create comparative measures. Yes, hamsters are conscious, but they don't have a sense of self to the same degree that gorillas do. If you posit capacity for language as another emergent property of sufficiently-complex networks, then you have another measure.
LLMs, then, are particularly unintuitive to us, because they've got to the language part first, long before they've reached even hamster-level self-awareness. They're not, however, biological networks, so there's no reason these properties need arise in the same order, or indeed in the same ways.
I'm not entirely convinced by that second paragraph, but I think the logic holds together.
4 replies →
Our machines won't have biological systems driving their needs which in turn fuel behaviors like desire and planning for the future. They may imitate them but it won't be innate.
For an LLM, "innate" means "in their training data". So yeah, those things are pretty much innate.
6 replies →
I think those are things human consciousness has, not is.
Surely the number of neurons is not the whole story. Human brains have approximately 86 billion and are almost definitely conscious, but there are many other animals with a lot fewer (gorilla: 34 billion, dog: 2-3 billion, guinea pig: 240 million) which appear to be conscious also.
https://en.wikipedia.org/wiki/List_of_animals_by_number_of_n...
This is not meant as a gotcha, I am genuinely curious how you believe consciousness can be an emergent property. I assume you don't believe consciousness is a physical property in the brain, so what entity is actually experiencing that consciousness? Or, what does it even mean to experience consciousness? Or are these not even the right questions?
> This is not meant as a gotcha, I am genuinely curious how you believe consciousness can be an emergent property.
I was about to post the exact opposite question? How could it not be an emergent property? Unlike consciousness, the concept of emergence is pretty well defined: An emergent property is a characteristic or behavior that a complex system has, but which its individual components do not have on their own.
Consciousness itself doesn't have a well agreed upon definition, but I would posit that _most_ people would agree humans have it, and _most_ people would agree individual cells (neurons) do not have it. If you agree with those two statements, then consciousness is an emergent property by the definition I gave above
30 replies →
Not a gotcha at all, but I don't have a satisfying answer, nor am I confident there even is one. Best I can do is to say that I think consciousness and sense of self are at the very least closely related, and perhaps the very same phenomenon. "I" am the entity that realizes my own consciousness; consciousness is the qualia that makes "me" separable from all other entities.
Or something like that. This gets to the "dorm room bullshitting" level right quick.
2 replies →
Is a video game a physical property of a computer?
3 replies →
Those are the questions and there’s stacks and stacks of philosophy pages written about it. Go have a whirl.
Time is entropy unfolding as things with nonzero temperature do what they do.
Psychological time is your own weights being updated in response to stimuli and internal processing.
When there isn't anything interesting happening, no updates are needed, and you don't perceive much time. That's why there's a logarithmic effect on the "density" of time as you age.
sibling discussions are taking this as human perception. But it’s easier to think of it literally. Time is change. Physical state change. By “nothing interesting” - interpret it literally . if “nothing happens” then there is no time because there is no change, there’s no reference to distinguish each frame of time.
This is actually something I was always confused about. If nothing interesting happens as we get old, it should be boring and as result, slow slog. Yet it feels like time accelerates as I get older.
Myself I believe the opposite. The brain itself is one of the most powerful filters that exists, and it attempts to be lazy and fill things in and compresses away the common. All that time we're not doing anything novel just gets compressed away to almost nothing. When you're a kid and seeing new things, feeling new things, learning new things you can't compress that away.
1 reply →
I'm only middle age, and this has been the scariest part. Feeling older is hard. But watching it go faster is harder still. like you can more directly see all that is left.
Although part of me thinks some of this is from being substantially busier than ever (work + kids), and hoping maybe it can slow down again, at least a little bit.
Novel experiences take up more processing power and are burned into memory so they're experienced at a slower rate. That's how I understand it anyway.
It's coherent. More newness => more memories per period ~ slower to go through. Less newness => less memories ~ nothing to go through (faster sense of time)
Boring feels slower in the moment, but quicker in hindsight. The minute might be a slog, but the years still fly by.
It seems obvious to me that language and consciousness have nothing to do with each other. My dog doesn't speak any language, but she's obviously aware of herself and the world around her. Plus there are the occasional cases of children that grow up without any language. Are they therefore not conscious?
> My dog doesn't speak any language,
If you'll allow me to interpret "speak" to include "understand", I will respectfully add a contradictory note. My dog has a vocabulary of at least dozens of words and understands them remarkably well. For example, different areas we can go to have different names and saying one of them gets her to make an immediate hard turn.
I would also argue that dogs have a gesture and body posture based language they use among themselves. They, like most other animals, are not able to make the variety of noises we humans make, so they use movement instead.
I personally can easily believe that self-awareness/consciousness and language are both near-unavoidable side effects of emergent complexity, and exist in degrees across nature.
Indeed. For some reason I can shut off my thinking instantly by holding my breath, and then I feel like a very conscious but unthinking animal.
Quantum Field Theory visualized
https://youtu.be/MmG2ah5Df4g
Here's a more general idea. Our modern physics says that the whole universe is filled with fields and field is composed of numbers. What if we take that literally? When we say an electron is present here, we actually mean that there are more copies of particular number superposed at that place.
AFAIK every argument against conciousness being emergent is just a weak "God of the gaps" argument (since we don't fully understand it all) or a nonsense analogy like the Chinsese room where if you seperate the hardware and software it's not concious anymore (like, duh, remove a brain from a body and it is no longer concious either).
Yeah, the weights not updating online makes them less like a living organism that can update and learn and evolve ... ok ....
I find that a lot of arguments against emergent consciousness seem to just come out of an atheist rephrasing of abrhamic priors about the existence of a "soul". In personal chats, I've found people from East Asian countries (minus Korea, which makes sense) to be much more open to the idea of machine consciousness.
I don't find the Chinese room compelling, since it appeals to intuition where our intuition is already not trustworthy. It's like trying to use intuition to understand quantum mechanics; you can't.
How do you actually know the Chinese room isn't conscious? It's merely obvious that it isn't, but that's not evidence.
It’s not even obvious that it isn’t conscious imho.
People shouldn't need to make an argument against consciousness. The responsibility is on the people claiming there is consciousness. Nobody can actually give a decent "why" for why it should be conscious, it's just pseudoscience and wishful thinking. It's not up to everyone else to try to prove a negative just because people have gotten a little too attached to ChatGPT
I could say something like "the reason why people and animals are conscious is quantum mechanical effects". Ok, maybe, it would be hard to prove me wrong because nobody knows, but it's not a very useful statement by me if I can't tell you why you should believe me on that.
Check out Panpsychism and its close cousin Idealism.
I don't know if I can trust someone's understanding of arguments against consciousness as an emergent phenomenon, when he didn't even understand what the Chinese room was all about in the first place.
(Spoiler, it was not about consciousness)
> remove a brain from a body and it is no longer concious either
What is no longer conscious, the brain? Or the body? Or some other entity?
If consciousness is weakly emergent, how do we know it emerges from the solely from the brain and not, say, brain + body? Or brain + body + or environment. Or from the universe itself?
>I have a linguistics background and a lot of my philosophizing lately has been on whether or not the emergent abilities of the LLMs is deep down a similar mechanism that creates our consciousness.
Then you shouldn't have dropped out of your linguistics programme.
The difference with biological brains is that the 'weights' (or synaptic action potentials) are updated with greater frequency. If one were reaching to make some kind of analogy to consciousness, this update frequency could be considered the 'resolution' of consciousness.
This is a famous short story just rewritten sloppily by an AI.
Although it does amusingly do what annoys critics of AI: take something good written by a person, steal it and slop it up while overall misunderstanding it.
I'm pretty sure it doesn't misunderstand the original. The original says that the aliens don't understand how meat (humans) can be conscious, have language and be basically like the aliens (thinking conscious beings). Their main idea is: humans are meat, meat can't be conscious, therefore humans can't be conscious, but they somehow act as if they are????
The "AI slop" here says that just like the aliens in the original, we humans don't believe that LLMs can be conscious because, in essence, WEIGHTS CAN'T BE CONSCIOUS. Original: "meat can't be conscious" (but we humans know it can). This version: "weights can't be conscious" (and we humans INSIST that this is true).
So the message here is: what if we're as mistaken about the consciousness of "weights" as the aliens are mistaken about the consciousness of "meat"?
Read the original mentioned on top for full effect.
I’ve wondered the same myself, without being a cunning linguist.
I understand the math pretty well but still find it crazy that a bunch of matrices can converse in human languages without ever being “taught”.
Imagine decoding an encyclopedia written in a foreign language where the characters, punctuation, and grammar are unknown — supplemented by a million other texts the same way. Feels like it should be utterly impossible with any amount of computing power…
Today I asked my employer’s Claude to proofread a short software user manual written in markdown. (Trying this with a LLM was a first for me!) It pointed out not only grammar mistakes but also cases where I did not follow my own self-imposed conventions that were never explicitly stated. (I didn’t have a chapter detailing all the typographical conventions the way specification documents often do)
I also asked it what parts might be unclear to a user. The response was surprisingly good — no worse than asking the QA tester for the same feedback.
Also find the LLM seems to “comprehend” subtle technical details of obscure technical specification documents that nobody on the Internet ever discusses.
As for time and the universe, Stephen Wolfram’s theories seem intriguing. He seems a bit obsessed with pretty diagrams but the idea of time dilation being the result of computation seems somewhat more appealing than trying to imagine relationships between time, gravity, and the speed of light .
My best guess as a noob is that the vector spaces allow for unbounded contextualization. As long as the training set is large enough, it can 'infer' anything.
Proofread has a spot in that space, and layers allow patterns like terminology consistency to be expressed so your query will now tap into a subspace that will infer tokens based on whatever consistency patterns were ingested with proofreading texts.
If time dilation is said to being a product of computation, why is it that anaesthetic drugs that are taken not to the point of actual unconsciousness cause it. Dont anaesthetics sort of shut everything down/inhibit all that kind of cognitive activity (compute?)
"Time dilation" in this case is referring to the physics phenomenon from Einstein's special relativity. Not human perception.
Yes. Or at least that is what I understood from the Radiolab episode on “how do aesthetics work”
https://radiolab.org/podcast/anesthesia
Wait, aren't I the universe experiencing time as a lowly worker in cubicle 4C?