Comment by hexapus
2 hours ago
I think it's good to continually remind oneself that these are next-word/token-predictors...you can wrap them up in a harness, you can feed them into each other, you can influence the likelyhood of a given word through weights and system prompts...but that's literally all they can do. They can't reason, but they can predict what reasoning looks like.
None of this means they can't be useful tools, but when they fall down, we should remind ourselves of why that is, and be honest with ourselves about what they could ultimately be capable of.
When you use "next word predictor", it sounds like slow process, like each word take few seconds to minute to be generated. Why not call it "sentence predictor", or "paragraph predictor" ? i would argue it's the same in principle.
It's like calling a video clip a "picture sequencer".
It’s equally useful to continually remind yourself that a Turing machine is just a spool of tape with a magnetic head, and that a human brain is just fatty lipids in a chemical bath.
What is the difference between reasoning and predicting what reasoning would look like?
The same as the difference between a thing and a statistical model of a thing.
They don't always match up.
In theory, theory and practice should be the same; in practice, they are not.
Human reasoning also does not match up with reality. Isn't it statistical modeling all the way down?
2 replies →
One has a human behind it and one has a matrix latent space. Usually human understanding is more "real"
Stallman vehemently states llms are bullshit generators with no true understanding.
But sometimes you can approximate your problem and unfold an llm interlocator to producing the strings you need for your task