Comment by pixl97
2 years ago
>it doesn't know what any of the data is. It only knows where.
And this right here is already driving a nail in the Chinese Room problem. At least from my interpretation of the problem that Searle presents, digital computers should not be able to do that at all, and yet here we are.
The situation isn't that mysterious or unknowable.
It's English that knows English. Chinese knows Chinese. The essence of grammar is encoded in the grammar itself: recursively.
Imagine the slabs of concrete that make up a sidewalk: between each of the slabs is a crack. Some slabs are shorter than others, so the distance between cracks isn't consistent.
Now imagine you took a string of pictures, each 1ft apart, all the way down the sidewalk, then stitched them together.
You show your friend the pictures. What do they see? A sidewalk.
ChatGPT gets a string of tokens: each token a few characters from the training dataset's text. That text is given in order. The boundaries between tokens are not in the same place as the boundaries between words, but they line up just as neatly.
Now imagine you shuffled the pictures, then stitched them back together. Does it still look like a sidewalk? Close enough. Some cracks are too close together or far apart to make sense, though.
With a handful of pictures, our sidewalk can go forever. And we can look at the original order to see what looks right and what doesn't.
If we avoid placing cracks closer together or farther apart than we saw them in the original, our sidewalk will look pretty good. If we try to repeat the original order, that's even better.
That's what ChatGPT does: it repeats what it knows in the order it has seen. The objects it is repeating are tokens, not words; but you can't tell that from the result.
But repeating text in "semantically familiar order" is how language is structured. Even if we didn't find or recognize words and subjects, we still get their effect, because the language already put that significance into the semantic order.
ChatGPT would be a meaningless continuation of nonsense if it wasn't trained on text that already contains language. But it was! Every token is a handful of meaning, neatly scooped out of a presorted list of semantic data. That order is preserved.
Even if the boundaries are drawn in the wrong place, the result looks just right, and we can see what we want to see.
>ChatGPT would be a meaningless continuation of nonsense if it wasn't trained on text that already contains language
I mean, if you put a child in a room and severely neglect them, you don't get a child that can speak any form of human language, and you'll find they an extremely underdeveloped brain.
If you put your pet rock in a room and severely neglect it, it will fail to develop human language. That doesn't mean you can expect it to.
ChatGPT isn't magic, and it's not a person either. It's a language model. Specifically, it's an implicit model of semantics.
If you gave it pure random data to model, it would; and that model would be useless, because it is only patterns of noise.