Comment by aeve890

2 days ago

>I don’t understand how it can know that a token is correct.

It can't. The next token is just the most statistically probably given the context (at least in transformers). Try a very small/weak model in your own machine and more often than not it would get stuck repeating the same word or even just output garbage. Because after training and quantization (where some information is lost), that's the most probable next token. Large models can be tricked to fall in the same behavior with very very specific inputs. Still happen, even in frontier models. And they can't detect if the output is wrong.

That's why the premise in TFA is wrong, because a transformer is a next-token predictor. It literally is that. There's nothing secret or magical, it's just a very mechanical process, with a lot of matrix multiplication, normalization, a few random passes, mappings between embeddings and a dictionary of tokens, in a very very high scale.

If someone has found something that's not a mechanical, algorithmic computation and llms are doing something nobody can explain and can't even be modeled in math, I'm happy to be educated.

5 comments

aeve890

Reply
major__4  2 days ago

classical particles obey deterministic state evolution rules, yet put 10 in a box and you cannot say where they will be 5 minutes later.

  • aeve890  2 days ago

    Your point being? llms are software running in a fixed pipeline (barring variance induced by randomness in some layers). There's nothing like decoherence/thermal effects in a "lower level" that could induce stochastic behavior in a higher level. There's no Brownian motion in llms, if that's your analog with classical mechanics. You might argue emergent behavior that would look ordered/intelligent in some way and nature's full of examples of that but we don't attribute intelligence to physical processes.