Comment by throwaway71271
7 months ago
> I disagree with that strongly. The LLM is obviously not a human or a person, but it's not a trivial token predictor, either.
I am sorry, by no means I think it is a trivial token predictor, or a stochastic parrot of some sort. I think it has a world model, and it can do theory of mind to us, but we can not do theory of mind to it. It has planning as visible from the biology of language models paper.
> So again, I disagree about AI-generated tokens having no meaning. But I would agree there is no human connection there
What I argue is that language is uniquely human, and it is how it is because of the human condition. I think we agree more than we disagree. I say that the meaning is 'halved', it is almost as if you are talking to yourself, but the thoughts are coming from the void. This is the sound of one hand slap maybe, a thought that is not your own but it is.
I guess I am saying is that AI is much more like Alien than Artificial, but we read the tokens as if they are deeply human, and it is really hard for people to not think of it as human, purely because it uses language in such profound way.
Thanks for clarifying; indeed, I think we actually have somewhat similar perspective on this.
> I guess I am saying is that AI is much more like Alien than Artificial, but we read the tokens as if they are deeply human, and it is really hard for people to not think of it as human, purely because it uses language in such profound way.
That's something I keep thinking about, but I'm still of two minds about it. On the one hand, there's no reason to assume that a machine intelligence is going to be much like ours. You put it nicely:
> it can do theory of mind to us, but we can not do theory of mind to it.
And on the one hand (still the same hand), we shouldn't expect to. The design space of possible minds is large; if we draw one at random, it's highly unlikely to be very much like our own minds.
On the other hand, LLMs were not drawn at random. They're a result of brute-forcing a goal function that's basically defined as, "produce output that makes sense to humans", in fully general sense. And then, the input is not random - this is a point I tried to communicate earlier. You say:
> What I argue is that language is uniquely human, and it is how it is because of the human condition.
I agree, but then I also argue that language itself implicitly encodes a lot of information on the human condition. It's encoded in what we say, and what we don't say. It's hidden in pattern of responses, the choice of words, the associations between words, and how they differ across languages people speak. It's encoded in the knowledge we communicate, and how we communicate it.
I also believe that, at the scale of current training datasets, and with amount of compute currently applied, the models can pick up and internalize those subtle patterns, even though we ourselves can't describe it; I believe the optimization pressure incentivizes it. And because of that, I think it's likely that the model really is becoming an Artificial, lossy approximation of our minds, and not merely a random Alien thing that's good enough to fool us into seeing it as human.
Whether or not my belief turns out to be correct, I do have a related and stronger belief: that language carries enough of an imprint of "human condition" to allow LLMs to actually process meaning. The tokens may be coming to us from an alien mind, but the meaning as we understand it is there.