Comment by nl
4 years ago
> Translating ambiguous symbols to meaning and combining them in meaningful ways is a big deal which, apparently, these AI models cannot do.
Plenty of AI models do exactly this. Very clear examples include question answering models and code generation. In both cases novel, meaningful responses are generated.
> They can just parrot things.
That isn't true. While language models can parrot things it is generally special conditions that make them do it. Specifically, the conditional probability of the next character (or BPE or word depending on the model) has to be much higher than anything else which happens when the thing being parroted is unique text.
If you ask most Americans or a language model what word comes next in this: fourscore and seven year.. they'll give the same answer, for the same reason.
So is in your opinion General AI solved? Because reliably turning symbols into meaning, outside narrow or special cases, is General AI.
In my opinion, it's not solved. GPT-3 is not General AI, it's a more clever mechanism for parroting back text it cannot truly understand. Comparisons to ways humans confuse themselves are a red herring in my opinion: the old ELIZA program could reply like a very confused or trollish human would, but nobody would argue ELIZA was a general AI.
It's just that GPT is a fascinating and more convincing illusion than ELIZA. Unlike ELIZA, it can also be used for meaningful purposes.
I don't think "general intelligence" is a bright-line, but instead is a continuum, and I don't agree with your definition (although I appreciate you do at least give a definition).
I think that in general most human decision making is just pattern matching (plenty of evidence for this - read "thinking fast and slow" for an overview).
I think the extrapolation that ML models can do is a form of intelligence. I also think that the compression and encoding of inputs into a lower dimensional space is exactly the "turning symbols into meaning" that you call for in your definition.
Thanks for the conversation.
We obviously disagree. I don't think we are near general AI (and yes, I know the objection that everyone who says this is simply moving the goalposts-- regardless, I'm unconvinced). I think GPT et al are very interesting tricks, but still not general AI; and that the path to it doesn't lie in this direction.
I subscribe to the view we think of the human mind as a pattern matching computer simply because this is the current major tech, much like people in the past thought of "humors" or "steam machines". I think some of the analogies are useful, to a point, but I don't think there's hard evidence the mind is like a neural net (irony notwithstanding) or a pattern marching GPT-like algorithm.
Re: Thinking Fast and Slow, I see there are serious doubts about the validity of the book's foundations and conclusions, and that it's been challenged.
3 replies →