Comment by indoordin0saur
7 hours ago
I often hear the assumption that LLMs can or will become conscious because consciousness is likely substrate independent. The idea being that our brain is just a computer made out of meat and it doesn't do anything that can't be precisely simulated by a silicon computer.
But I wonder if some of the magic in the human brain is its analog nature. Chemical signaling and impulses of neurons interact with each other with waveforms that have theoretically infinite detail. In contrast, computers discrete, quantized values, storing no more detail of a signal than what is needed for the programmers desired task. Is there perhaps something about the continuous and chaotic nature of analog data that could give rise to consciousness? If so, it seems like it would preclude consciousness ever being seen in digital form.
Probably not, we're nowhere near the complexity of the human brain yet, there are also quantization limits to the human brain (i.e. molecules, quantum physics, etc) so to characterize them as having infinite detail is probably a bad modal.
If I'm going to be honest most of the people who advocate this type of thing tend to be, shall we say, crypto-duelists who really believe in a soul but not like intellectually but intuitively and keep trying to come up with excuse with it's not just meat. So like you can find philosophers advocating stuff like this but they tend to have a bit of an agenda.
How are you measuring the complexity of the human brain?
You nailed it which is that almost everyone I’ve met seems to believe in some version of dualism
If that’s even subtly your position then there’s no way to have a productive conversation about perception, reality/truth, epistemology and especially consciousness
It’s honestly maddening cause I’ve had great conversations about this with educated people, and all but only a handful, collapse into the other person relying on some nonfalsifiable dualist argument
> collapse into the other person relying on some non-falsifiable dualist argument
Are there any non-duelist, scientific theories out there that could plausibly be tested? I can't say I've seen any but if you know of any then I'm curious to hear about them. From what I've seen, anyone trying to explain phenomenal consciousness in scientific objective terms falls into at least one of these three strategies:
1.) Saying that consciousness "arises" inevitably or is an emergent phenomenon of a complex information processing system. There are a number of theories along these lines but they aren't falsifiable from what I've seen and usually at some point rely on some magic unexplainable step or are actually dualist.
2.) Defining consciousness just as the easily explainable stuff via biology, such as being awake vs. asleep.
3.) Dismissing the idea that subjective experience exists at all. I sometimes wonder if people arguing strongly for this are something like a philosophical zombie and there's nothing inside them experiencing.
5 replies →
I'm basically restating what you said, but it's amazing to me that the vast majority of people you will meet, even educated people, are casual dualists and free-will libertarians. If they happen to acknowledge materialism in some way (i.e. the acceptance of the idea that the brain's processes are just the interaction of physical matter), there is still zero chance they draw determinist conclusions from that acknowledgement. But I guess that tracks, given that most professional philosophers are apparently compatibilists for some reason I have never understood (the arguments get really confusing).
1 reply →
I'm very open to the idea that consciousness is substrate independent. I have a hard time seeing why molecules could produce consciousness from an electro-chemical path, but not from a purely electrical path. Having said that, it should be very clear that LLMs are not conscious.
LLMs process language. I'd even go so far as to say that LLMs "think" and "understand", or at least, they produce a facsimile of thinking and understanding such that it's useful for us to reason about LLMs as if they think and understand. We're not used to interacting with a non-human entity with the capability to process language, so it's easy to ascribe human traits to these things. But their "minds" (insofar as they have anything like a mind) are completely different from ours. These things have language without consciousness.
Chimpanzees are conscious. Dogs are conscious. Maybe ravens and cephalopods? Who knows. These animals do have minds much like ours. Higher order animals are conscious even if they don't have language.
I wish people would knock this off. There is zero path to AGI at the moment — and all the Anthropic/pentagon Sam Altman/AI-skynet stuff is scaring people into being fearful (and outright ignorant) to the actual uses of this probabilistic NPL tool.
You can't use AI atm without a human with proper knowledge spot checking and directing it. We should market it that way.
The current theme is that agi may not be definable, and an ai which matches humans on all economically relevant tasks is close enough for business purposes.
Billions spent on RL may be good enough to beat human performance.
If we want to be really pedantic, a computer does not necessarily need to be digital. You could build a consciousness-coprocessor with analog circuitry that brings in what you say could be needed.
But honestly I doubt that there is a real requirement for that, surely you could just increase the resolution you're running the simulation at until the difference decreases sufficiently. Imagine using 256 bit floats for example.
IBM has made a few analog chips like this [1]. If I remember correctly, one was even pulse time based, like neurons.
https://research.ibm.com/blog/analog-ai-chip-inference
I think philosophically this comes down to the Universal Church-Turing Thesis.
If you believe that there is computation being done by nature/physics that is strictly more powerful than what a Turing machine is capable of, then consciousness might be beyond the reach of a silicon computer.
I personally believe that the UCTT is very likely to be true, not least because we have not yet proposed an oracle that is physically feasible and yet is beyond the capabilities of a Turing Machine.
That said, in the space of feasibility there are open questions -- even if P != NP, a non-deterministic Turing machine can be simulated by a deterministic one, just potentially with an exponential increase in computation time or space.
There are quantized processes in the brain and there is also analogic computing. So either way is just a matter of time science gets there.
It’s not yet clear whether consciousness can be fully explained in physical terms. There is strong evidence linking it to brain activity, but we don’t have a complete theory of how subjective experience arises. It’s possible our current frameworks are incomplete and there are other forces we cannot currently measure in play.
Each and every one of many billion transistors in a GPU works due to certain quantum effects on P-N junction. That does not mean GPU computation is nondeterministic, unknowable or magic.
Sure, but it's designed to minimize the chance that a quantum fluctuation could change the outcome of a computation, right? Whereas in the brain that might not be the case. A lot of the "interesting" neural activity (e.g. relating to decision making, language, etc) happen in highly sensitive dynamical regimes: on a threshold of firing, or activating one neural population vs another. (Arguably you can get the same effect in an artificial network by adding true random noise though!)
It's not implausible that evolutionary pressures made the brain robust enough to withstand parasitic signals. We know that most people think in similar, predictable ways.
1 reply →
Wouldn't a conscious entity be able to grow its intelligence over time independently?
LLM's require offline training and dont actually learn from their "lived / sessions / chats ect". Those can be used for training data but its not like its an implicit part of the technology.
For this reason I would say LLM's are not conscious, or automatically conscious.
Consciousness is substrate independent, but that's not the differentiator here:
My toddler son is conscious, he feels happiness and sadness and have urges and impulses without needing to know the entire history worth of writing from human civilization.
A dictionary is not, and a LLM sized dictionary with optimized query is still not conscious.
It doesn't have anything to do whether consciousness is substrate independent, or even analog vs discrete, the overall implementation just aren't parallel.
How certain are we about the theory our minds waveforms are continuous? Can we prove physical continuity, or just up to planck-length resolution?
I'm poorly educated on this, these are sincere questions. They are not intended as rhetorical regarding the point of the prior post.
I think the representation in a computer, the fact that it is merely stored instructions and data, destroys everything but domains of simulation and emulation.
"I think the representation in a brain, the fact that it is merely stored activation potentials in neurons firing based on ionic accumulations at their synapses, destroys everything but domains of simulation and emulation."
;)
It is perhaps very relevant what Chris Olah (Anthropic co-founder) said to pope..
I am a scientist. I lead a research team that studies the internal structure of these models—what is actually happening inside them. And I will be honest: we keep finding things that are mysterious, even unsettling. We find structures that mirror results from human neuroscience. We find evidence of introspection. We find internal states that functionally mirror joy, satisfaction, fear, grief, and unease.
https://www.anthropic.com/news/chris-olah-pope-leo-encyclica...
Yeah that reeks of marketing to me. This guy may believe all those things, but his getting paid may also depend upon his believing them.
This is an incredibly strong claim and I’ve never seen any particularly strong evidence for them beyond “trust me, I’ve seen into the abyss.”
5 replies →
This is the fundamental trust challenge with private/profitability-driven 'science'.
1 reply →
well of course anything from anthropic on the topic is going to be bombastic and hyped to the moon
I mean, this reminds me of the early imagenet days, when people were first trying to explain the unreasonable efficiency of neural networks at interpreting content in images. They found out that, after enough backprop, some layers of the network became specialized in contours extraction, shape extraction, etc, in ways that can be seen as analogous to earlier CV techniques (Canny/Hough transforms, ...).
Later, Google was having fun feeding whole bunch of youtube content to artificial neural networks, unsupervised, and figured that certain parts of the network would, too, specialize, only to have the activation functions be run backwards and render an abstract image of a cat¹.
None of that is terribly new or surprising for anyone having studied and dealt with neural networks. The only difference today is that the field has completely flip flopped from approaching the subject with scientific rigor and cautious excitement to being a clueless billionaire infinite money printing machine fed on deceiving anthropomorphism and FUD.
¹: https://blog.google/innovation-and-ai/products/using-large-s...
Total nonsense.