Comment by montroser
15 hours ago
Sorry, but they can think and reason.
They can have empathy too. Meaning, you can describe a circumstance with many parties involved, with different motivations and conflicting goals -- and it will tell you how these humans would likely feel about each other and the situation at hand.
Being able to describe how humans likely feel is not evidence of the model feeling empathy.
And what would be evidence?
I’m not sure, what would evidence of qualia even look like?
By that logic, a book has empathy.
Not the book, but the author, yes.
Not the LLM, but the training material, yes.
1 reply →
They can provide text which contains thoughts, reasoning, and empathy, but is that evidence of them having thoughts, reasoning, or empathy?
To paraphrase the Chinese room argument: Put me in a room with infinite time, paper, pencils, filing cabinets, and give me an English language list of instructions for doing all the neural-net calculations required to run an LLM. Slide your prompt under the door, and, after doing all the calculations, I will slide the next word out, and so I will repeat until the response is done.
In this thought experiment, I'm just following the instructions. So, in this case, what is doing the "thinking" or "reasoning" or "empathy" here?
The inputs going through the model. That act is what is doing the thinking and reasoning and empathizing. Same as when your brain feeds your sensory inputs through its neural network.
Your brain is just a big set of weights/neurons organized into layers that do back and forward propagation, turns out. You are just statistics surrounded by flesh and bones.
A sociopath can also pantomime empathy for their own selfish reasons simply by observing how others behave in similar situations, while feeling nothing special at all.