Comment by qarl2
2 days ago
Maybe I'm wrong - but I see LLMs are a "next-token predictor" as somewhat equivalent to brains are a "bag of molecules".
Both systems have emergent behavior that goes well beyond what would naively be expected.
2 days ago
Maybe I'm wrong - but I see LLMs are a "next-token predictor" as somewhat equivalent to brains are a "bag of molecules".
Both systems have emergent behavior that goes well beyond what would naively be expected.
Agree. It’s a very limited description. Yes, technically LLMs are like Markov chains or n-grams predicting the next token. But it completely ignores the compression and interpolation capabilities achieved with transformers.