Comment by mannykannot
12 hours ago
When John Searle first published his so-called "Chinese Room" argument (https://web-archive.southampton.ac.uk/cogprints.org/7150/1/1...) he included his responses to several counter-arguments that he had been presented with in prior discussions: the systems reply, the robot reply and so forth. Interestingly (as it turns out) no-one seems to have argued that this was a task that could be completed without any understanding, but now that AI technology has reached the point of being able to implement the Chinese Room, it has become the fallback position of those who insist that true AI is impossible!
This is not an argument for true AI being possible, let alone already achieved; it is a warning about depending on arguments from incredulity, which are particularly susceptible to future developments.
The problem with philosophy is that someone can present an argument that is pretty dumb, everyone can point out it's dumb, people can show evidence that it's dumb, reason through how dumb it is, and publish articles about the many different ways in which it's dumb, and still it gets trotted out as if it counts as evidence for some argument. Searle's Room was debunked immediately upon being suggested: the "Systems Reply" is the debunking. His response was: "but if you take all the knowledge out of the system and put it in the person, then the person still doesn't understand Chinese!" Which is obviously false and immediately collapses his entire argument down to nothing. He starts from the premise "this person doesn't understand Chinese; he is just following instructions that are written down for him" and then immediately rejects his own premise and say "actually he does understand all of Chinese and everything he's doing, but he still doesn't, you know, understand it, see?" It's completely embarrassing.
Searle's Rooms is a fully rejected and rebutted argument, and Philosophy will never be (and never has been) a serious discipline until it figures out how to actually let go of failed arguments.
Huh? How does the person having all the instruction books memorized and whatnot imply that the person understands Chinese? Memorizing instructions to the point that one can mentally follow them doesn’t imply understanding why they work. If the person doesn’t understand the correspondence between the Chinese written words and things in the external world, then surely they don’t understand Chinese in the sense we generally mean when we talk about whether a person understands Chinese. (I think this is related to the robot reply?)
That’s not to say that I think the Chinese Room Argument is convincing. I don’t think it is. Or, I guess it might demonstrate something, but not what it aims to show.
I would argue that if, say, the person who has memorized this program and mentally executes it, and if this program also includes actuating some levers to control a mech or something, and the program (running via the person mentally going through the instructions by rote) takes actions to pursue some goals (e.g. purchasing some wood by conversing in Chinese and building a tower with that wood), while the person on whom the program is running wouldn’t know how to order a cheeseburger in Chinese,
Then, the program running in that person’s head might well understand Chinese even though the person doesn’t. (Whether or not the program actually understands Chinese, I think probably depends on the specific of the program in ways that I don’t think anyone knows. I don’t expect that any program that would understand could be small enough that a human person could memorize it and execute it by rote in this way, but this is a thought experiment and such issues can be disregarded I think.)
The point is that Searle's entire exercise is to show that there can be execution without understanding, and he does this by creating an agent whose only purpose is blind execution of rules. Since this agent doesn't have the understanding, there must be no understanding.
Critics immediately point out: yes, there clearly is still understanding, it's just in the system itself -- the program. The fact that the person does not have the understanding doesn't mean the understanding is gone.
His response to this is: but what if the system was the person's brain!?
Okay, so either it doesn't address the criticism at all (there is still a system of rules that contains the understanding; who cares whether that system is internal or external to the person's brain?) or it completely destroys the original premise of the argument, which is that the understanding is not in the person! You can't put the understanding back in the human agent and still claim the human doesn't understand what they're doing! His entire argument relies on the fact that the human agent obviously doesn't understand Chinese, and then says, actually, their brain contains everything it needs to know to have completely fluent conversations in Chinese. So the person does understand Chinese!
> How does the person having all the instruction books memorized and whatnot imply that the person understands Chinese?
How does a person's brain containing all the facts, information, and skills that it needs to have a fluent conversation in Chinese imply that the person understands Chinese? How does it not!?
3 replies →
Searle's main argument seems to be that meat is magic and silicon is not.
The person who memorized the book doesn’t understand Chinese. In fact they don’t even understand that Chinese isn’t a language. Searle’s response stands.
So you could put this person in a room with a bunch of native Chinese speakers, and he could hold conversations fluently with all of them. Yet you claim he doesn't understand Chinese? How can you possibly defend that? This is very close to "zombie theory" -- everyone else is an automaton except you. Nobody else has real thoughts. That obviously fluent Chinese speaker doesn't really understand Chinese.
4 replies →