Comment by hexapus
3 hours ago
I think it's good to continually remind oneself that these are next-word/token-predictors...you can wrap them up in a harness, you can feed them into each other, you can influence the likelyhood of a given word through weights and system prompts...but that's literally all they can do. They can't reason, but they can predict what reasoning looks like.
None of this means they can't be useful tools, but when they fall down, we should remind ourselves of why that is, and be honest with ourselves about what they could ultimately be capable of.
What is the difference between reasoning and predicting what reasoning would look like?
The same as the difference between a thing and a statistical model of a thing.
They don't always match up.
In theory, theory and practice should be the same; in practice, they are not.
Human reasoning also does not match up with reality. Isn't it statistical modeling all the way down?
2 replies →
One has a human behind it and one has a matrix latent space. Usually human understanding is more "real"
Stallman vehemently states llms are bullshit generators with no true understanding.
But sometimes you can approximate your problem and unfold an llm interlocator to producing the strings you need for your task
It’s equally useful to continually remind yourself that a Turing machine is just a spool of tape with a magnetic head, and that a human brain is just fatty lipids in a chemical bath.
When you use "next word predictor", it sounds like slow process, like each word take few seconds to minute to be generated. Why not call it "sentence predictor", or "paragraph predictor" ? i would argue it's the same in principle.
It's like calling a video clip a "picture sequencer".
It's literally what a LLM is, and it's not even a word it's a token which is about 2/3'rds of a word. It has nothing to do with speed of producing tokens. Others are not responsible for correcting your bad understanding of a true statement.