Comment by ACCount37
5 hours ago
A lot of people who "know how technology works" just went looking for copium, and found some. Now, they "know" a comforting lie - something like "it's just next token prediction".
Very comforting, that, but actively harmful to understanding.
The understanding starts with: we don't actually know how LLMs do what they do. They're more grown than designed. And it only gets worse from there. Very little comfort to be found in modern AI.
There are two things here: one is how an LLM is fundamentally structured and designed, the other is how an LLM distributes and 'lays out' the relationship between inputs and outputs through layers and weights.
We might not know how the actual distribution works, but we do know how it i s fundamentally structured and designed -- because we did it. We also know that there is something like a representation system inside them. And we also know that human beings do not hold 'internal representations' like any AI system needs to. So there isn't any 'intrinsically magical' in modern AI systems.