Comment by pessimizer
4 years ago
That's because they're not modelling anything. The shocking thing about current AI models is that just sort of repeating and copying from memory what you've heard and seen gets you 97% of the way to imitating a person.* They still need to generate actual models somewhere to create consistency; so many generated images with one eye completely different from the other, or three arms, or fingers that grow into their cellphones.
If you solve this, you've probably solved almost anything in the simulation field. I have no confidence that the solution will even be complicated. Information consumed needs to be used to add to some sort of model, and that model always needs to be used as part of input. The complicated part would be to make that base model able to modify itself reasonably based on input, to tolerate constant inconsistency, and to constantly refine itself towards consistency i.e. ruminate.
I think a huge difference (which I think was approached through theories of embodied cognition) is that people start with a model (or the ability to create a model) of themselves. We can apply that model to other things and use it both to change how we ourselves behave, and how we speculate about the invisible states of other things. It's not for nothing that we can (and must) anthropomorphize anything.
-----
* Which was huge towards the confirmation of my belief that this is all people do 97% of the time.
This is factually wrong, both in terms of quantity and quality.
Current AI models are not "just sort of repeating and copying from memory". This is just an incorrect characterization of how they work and how they perform.
AI skeptics often say things like this then backpedal with something like "Well they aren't really repeating what they heard, but their generative model is just a slightly more sophisticated version of repeating what they've heard." But this weaker claim is also true of humans. It's certainly the case that >97% percent of what humans say is "just repeating and copying" in the same sense.
> Current AI models are not "just sort of repeating and copying from memory". This is just an incorrect characterization of how they work and how they perform.
You say this, but don't explain how. Because this is exactly what they are doing.
> AI skeptics often say things like this
I'm not really an AI skeptic. I think that we're very close to AI being indistinguishable from people. There are clearly problems that need to be solved, but I think the hardest problem was accepting the fact that humans are largely just copying and realizing that would be enough to get you 97% of the way there, especially if you gave a machine far more to copy than a human could consume.
> then backpedal with something like "Well they aren't really repeating what they heard, but their generative model is just a slightly more sophisticated version of repeating what they've heard." But this weaker claim is also true of humans. It's certainly the case that >97% percent of what humans say is "just repeating and copying" in the same sense.
Maybe I'm not expressing myself clearly, but it seems that you're just repeating my comment with a sneer. Agreeing angrily?
I'm disagreeing with the language you are using to characterize models. "copying from memory" implies that there is something being copied, and a memory that you are copying it from. I am pointing out that LLMs do not do this. It's not how they work.
If you polled 1M random English speakers randomly and asked them whether or not a system that "just sort of repeating and copying from memory" could produce completely novel answers in response to completely novel questions, I suspect that the overwhelming majority would respond by saying no.
Similarly if you asked 1000 people working on LLMs whether they work by "copying from memory", I suspect nearly all would say no. It would be accurate to say they are "generating text via a probabilistic model of language, which is encoded in the weights of a neural network", but there really is just no sense in which the models are "copying" anything.
That being said, these models do "copy" some text in the sense that they can reconstruct some strings from their training input. For example every LLM I have played with can recite the first few paragraphs of A Tale of Two Cities verbatim. But that's a capability they have _in spite of_ their actual design, not because of it.
7 replies →
can you actually share what "current AI models" are then? Not trying to be rude, but you just said "na ah" and then refused to argument any position.
Current LLMs are "modeling" something according to pretty much any sense of the word "model".
In the technical, computational linguistics sense, LLMs are language models that give a conditional posterior distribution over sentences. Given some (constrained) context, the model tells you the posterior distribution over sentences in or around that context.
In the nontechnical, layman sense of the word, they are a system that is used as an example of language. LLMs imitate language by generating new sentences. They are a "model" in the same way that an architectural model is a model, or in the same way that a statue is a model of a human.
The other point I disagreed with is the characterization that LLMs "just sort of repeat and copy from memory". I went into more detail about that in other replies.
2 replies →
There is a reasoning loop that LLMs are clearly missing.
"repeating and copying from memory what you've heard and seen gets you 97% of the way to imitating a person."
I'm in this hot take and I don't like it..