Comment by paulluuk
13 hours ago
> These theories are flawed in the sense that they cannot account for subjective experience and agency
So, I work in AI research (as a research engineer though, not a scientist). I've also studied philosophy and I'm a vegan. Yes yes, insert "they will tell you" joke here, but I promise it's actually relevant this time.
First, while I studied philosophy one of the things that stuck with me the most, was the discussion of "souls": humans have souls, animals don't. For centuries the specifics of souls were discussed: people would be weighed while they died, in an effort to measure the approximate weight of a soul as it departed the body. Discussions about how many souls (or angels) could dance on the tip of a needle. Many people still believe in souls, but it's very hard to have a real discussion about them because by definition they do not "interact" with this world in a way that can be measured.
When discussing whether it's okay to harm animals for food or sport, one other argument I hear quite often (other than having no souls) is that animals do not experience "qualia": basically the smallest unit of "subjective experience". People know that they themselves experience qualia: the sensation of touching a doorknob, the taste of fresh fruit, the sense of beauty watching a rainbow. Ironically, they would say that animals are like robots: just (biological) machines acting on instinct, and feeling any kind of compassion for them means you are anthropomorphizing.
Subjective experience (or at least qualia) and souls both have one thing in common: they can not be measured externally in a meaningful way. You can simply state that an AI system no matter how advanced, has no soul and has no subjective experience. And that's pretty much that. There's no meaningful discussion to be had about it, because no matter what an AI might tell you: it has no way to prove it to you. In fact, you can't even be certain that anyone other than yourself has subjective experiences: you assume that because they are humans like you, and you experience them, that they probably do as well. They tell you that they do. But a human without subjective experience, someone on "autopilot", would be absolutely indistinguishable from a human who does have them.
But perhaps I am conflating here whether experience can be "measured", with whether a system even allows experience in the first place regardless of whether it can be measured. I think that Dreyfus and others argue that in order to have any "experiences" at all, you simply must have a body in the real world, and you must care about that body. Please correct me if that's the wrong interpretation, I haven't actually read the book. That argument would be harder for me to discuss, because I personally believe that consciousness will "emerge" from a complex interaction of relatively simple systems - but that's also just a theory. I don't believe that experience is literally impossible to engineer, as consciousness has emerged from non-conscious being through evolution, so clearly there must be some kind of mechanism for it -- and if there is, then I believe it can be replicated, we just don't really understand it well enough yet to do so. And with how AI tech is going, I think that we're more likely to accidentally stumble upon it than we are to get these deliberately.
“Reflections on trusting trust” is the paper that posits a compiler which is edited once so that when it compiles a program, it adds a security vulnerability to it, and when it compiles it’s own source code, it adds this edit into itself. Then it is used to compile its own source code once. Then this edit is removed.
Now any study of the program or compiler source code will not show any vulnerability, but compiling the program will make a vulnerable program, and recompiling the compiler from its clean source code will not fix the situation.
This carrying down of a pattern which is not written down anywhere, a flaming torch lighting a torch lighting a torch, is analogous to four billion years of life on Earth. We talk like DNA is an everything-code that defines a human and a human brain, but it’s the implicit behaviour of cells (‘compiler’) and the mechanisms inside them which interpret DNA. The unbroken chain of life getting more and more complex and never being restarted from scratch, with the behaviours not written down anywhere for us to study. How does DNA arrange for x, y, z to happen? Maybe it doesn’t at all.
Accidentally stumbling on a mechanism that is simple enough to be recreated with every human birth might be possible, accidentally stumbling on a mechanism that took evolution billions of years to find and which it has hung onto by copying it and has never recreated it from scratch, could be much less likely, in a much bigger search space.
> Accidentally stumbling on a mechanism that is simple enough to be recreated with every human birth might be possible, accidentally stumbling on a mechanism that took evolution billions of years to find and which it has hung onto by copying it and has never recreated it from scratch, could be much less likely, in a much bigger search space.
Maybe, but you could make the same argument about anything artificial.
I don't know what point you're making; I'm making the point that it might be harder to discover what makes humans human, than is often suggested. You can't make the same claim about "anything artificial", we know how to dry muddy clay into clay bricks and stack them into a brick wall, and that can be taught from scratch to new people in hours.
You can make a similar argument with a company like ASML where their secret sauce is the organisational ability to fine-tune 100,000 components into a precision Silicon-wafer etching machine. You're far more likely to accidentally stumble upon "how to recreate a mud hut" than "how to recreate ASML". Okay, and...?
Vegan ML engineer here. In total agreement with you. People are just moving the goal post to keep themselves protected from the obvious conclusion: there is nothing really all that special about us humans. Perhaps subjective experience is simply the internal state of a self supervised continuous learning algorithm and we don't like that conclusion very much.
It's too bad it's so hard to pin down a definition, but in practice I feel like most animals with brains experience degrees of qualia. Some mornings after a night of poor sleep when I wake up super-slow I wonder if that's how animals experience thinking.
My biggest problem with "brains are machines" arguments is that there is a risk there is unknown physics at work that is not representable as a Turing machine. What if there is some quantum field effect powering everything?
They know quite a lot about how neurons work to the extent they can replace bits of brains with artificial retinas or cochleas and interface with devices like neuralink. It's unlikely there is a quantum field effect of the type you mention powering things, although of course atoms and the like obey quantum mechanics in the normal way.
Quantum field effects? You don't need these, IMHO, if you look at how highly parallel things seem to work in brains.
Marvin Minsky's theory of a "Society of Mind" describes a (highly) distributed model of the mind. Which BTW, always reminds me of the first Shrek movie, where the donkey jumps up and down, shouting "Take me! Take me!" to Shrek. That's similar to what I observe when I'm undecided but two instances of "sub-processes" (or agents as Minsky calls them) of my mind try to get attention.
Daniel Dennett similarly gives a distributed model of consciousness. Where many parallel "processes" are at work, competing and "observing" each other. And this parallelism is happening with a much, much higher degree than any of our computers parallelism.
2 replies →