Comment by frankohn

I find this kind of article unfortunate because it is very assertive in claiming things that are, not just untrue, but also confused about what consciousness really is.

Well, the problem is that there is little agreement about a widely accepted definition of consciousness and, in addition, this subject was actually left to philosophers, which is even worse because, in my view, they usually just produce a lot of nonsense in terms of definitions and arguments.

To me, a reasonable definition of consciousness is: a system which is capable of recognizing aggregate objects from a stream of sensory information it receives and which is capable of reasoning about the recognized object without immediately acting on it.

Well, what does "recognizing" mean? Merely that some of its neurons, related to the kind of identified aggregate object, are activated. These neurons, in a generalized sense, can be whatever things can work as neurons, just things that can be activated and propagate to other interconnected neurons.

For example, when we see through our eyes, we have an incoming stream of amorphous image information, but our brain can recognize that we are seeing a tree because we learned what a tree is, and when we see it, some neuron clusters activate to recognize a tree. In turn, when we recognize the tree, the thought propagates through our brain so that we are conscious of it.

In the same way, an LLM can perfectly recognize a tree from a stream of tokens — its sensory input, where the tokens describe a tree. The LLM will recognize that the tokens are describing a tree, and some of its "neurons" specific to the concept of a tree that the LLM had learned will be activated and will propagate through its brain. The fact that "neurons" are implemented as floating-point numbers for some parameters and connected just through a matrix does not mean they are not functionally capable of the same things; they are just implemented in a different way.

So the remaining part of my definition is "after recognizing an aggregate object, the thought propagates through the brain". The propagating part, to me, is just the very basic way a brain works: neurons are interconnected, and when some fire, other neurons fire, and that propagates.

In my opinion, consciousness has nothing to do with emotions or with survival. I do not see why emotion is necessary to consciousness; they are just different things. The author writes: "Without them [emotions], there is no conscious experience, only computation." But that makes no sense to me: the author has decided a priori that some things are "computations", and just because they are "computations", there cannot be "consciousness". But to me, this is a plainly wrong argument that does not hold.

I also do not see why the survival aspect would be needed for consciousness.

So to me, any recent reasoning LLM is conscious by the definition I gave, but also generally speaking. It is conscious upon a sensory stream of tokens: the LLM sees the world through tokens and expresses its thoughts through tokens; it does not mean it has no consciousness nonetheless. The fact that we do not give it a stable support to retain its memory and individuality is just a fact related to the way we build and use them, not about their intrinsic capacities.

Note: ChaptGPT came up with what is probably a better definition of my own: "A system is conscious, in a functional sense, when it can form internal representations of objects, states, or events from its sensory or informational input; make these representations globally available to many parts of itself; integrate them into a temporally persistent model of the world and of its own state; and use this integrated model for flexible reasoning, self-monitoring, and action selection independently of immediate stimulus-response behavior."