Comment by estearum
6 hours ago
> But brains do much more than just predict tokens based on previously seen tokens. I think all the other things brains do are probably important for our intelligence.
Like what?
What specific biological structure in the brain could be doing anything other than producing output as a function of 1) current electrical/chemical/thermal inputs and 2) previous electrical/chemical/thermal inputs?
LLM doesn't just produce output as a function, they are much more specific: they predict text based on text they have been trained on.
Yes that is the item (2) parameter described above.
No, brains doesn't just try to mimic pasts signals, LLMs do that but brains doesn't.
The way they make LLM solve problems is by adding a lot of logical jumps into its data, or break down different problems etc, and then as it predicts the text it predicts these logical jumps and then solves the problem. That is very different from how humans learn to solve problems, you don't feed them a billion different state transitions they have to encode to be able to navigate math, they learn to become proficient at math from a few hundred to a few thousand examples, that is fundamentally different from how LLM can learn.
That LLM are so slow learners that requires massive amount of data is a big reason its hard to make them smarter, and its caused by them being next token predictors. And the reason humans can learn with so little data is because we are not just next token predictors.
4 replies →