Comment by 6gvONxR4sf7o
4 years ago
With all this discussion about whether LLMs are “intelligent” or “really understand” it’s getting really obvious that the words “intelligent” and “really understand” aren’t very useful words any more (in this context).
Like, most of us agree about what LLMs can and can’t do and how they do it. So we’re just having fun arguing about how we’d prefer to define “understand?”
On the contrary, it's important to realize that these models have no sort of understanding in any definition of the word. We can always decide to redefine the word "understanding", but that doesn't seem like it would provide any sort of benefit. Perhaps outside of avoiding a difficult conversation about the (huge) gap between ML models and consciousness.
Strongly disagree. Mathematician Gregory Chaitin said "compression is comprehension", and I think this is exactly correct: something is "more understood" if it can be internally represented or externally reproduced with less information.
Based on how transformers are trained to predict the next token, you can view them as powerful compression algorithms [1]. They truly must - and do - understand complex concepts to generate text as they do.
Understanding isn't the full picture for intelligence. Marcus Hutter has argued that AGI requires both compression/comprehension, and being goal directed/having agency. I'd say chatGPT has advanced understanding, but zero agency.
[1] https://ml.berkeley.edu/blog/posts/dalle2/
There's the hutter prize which is related to that. I feel like it sort of missed the mark in terms of scale though: the compression techniques needed to compress a relatively "small" and specific dataset like Wikipedia are completely different from the techniques needed to "compress" the entire internet. It's only with the latter that we're seeing something interesting start to happen. The other constraint the hutter prize has is lossless compression, which isn't really conducive to general "learning"
Fair enough. I can see how for a limited domain of understanding, it can be a reasonable analogue for compression. I still don't see the benefit of reducing the domain of a word just to be able to put a new technology on arbitrary scale of "intelligence". I think we should be able to appreciate the power of ChatGPT-like models without lossy compressing of our vocabulary in the process.
Well the prompt is there to provide agency isn't it?
1 reply →
It's not that we don't understand that this is just a language model.
We do understand that.
The fact that a language model can produce convincing output hints that it is not that different from us.
Imagine something in 5 or 10 years time with 1000 times the power. What will it be like?
I know you think our awareness makes us fundamentally different. But what if it doesn't? What if we're just a machine that likes to anthropomorphize things reflected back on itself?
Ultimately, we won't have to know the answer.
In 10 years AIs like this will be in computer games. Some people will find the results of that disturbing. Is highly accurately similated suffering different to real suffering? The philosophical answer won't matter, the practical answer will be that AI use in games will become regulated. And then you have the first glimmer of AIs getting rights.
> In 10 years AIs like this will be in computer games.
Why do you believe it will take 10 years?
How do you explain how it knows the output of the first Python expression in this article, if not by understanding it?
Let alone the fact that’s it’s running in a simulated virtual machine, and the other examples presented in this article, which add layers and layers of complexity on top of that.
I can't imagine any useful definition of understanding that has no overlap with what GPT accomplishes.
I believe one would have to foray into Martin Heidegger for this, understanding not as an epistemic term but as an ontological category: "Understanding is… the original form of the realization of Dasein, being-in-the-world" [1]. And since GPT is not embodied, by definition it is not "under-standing". Heidegger uses again and again this language game of hypokeimenon [2], that which lies underneath, standing under, the subject.
[1] https://plato.stanford.edu/entries/understanding/#:~:text=Un...
[2] https://en.wikipedia.org/wiki/Hypokeimenon
2 replies →
I was thinking the same. Perhaps a better word to describe what it is doing is "following" our prompts decently?