Comment by shawabawa3
9 months ago
You can create a new game with new rules never seen before
You can explain this to an LLM
The LLM can then play the game following the rules
How can you say it hasn't understood the game?
9 months ago
You can create a new game with new rules never seen before
You can explain this to an LLM
The LLM can then play the game following the rules
How can you say it hasn't understood the game?
The LLM is only capable of doing so if it has encountered something similar before as part of its training.
Claiming anything else requires a proof.
the extraordinary claim would be that LLMs can only do things they've seen before exactly, given the compositional and emergent capabilities we observe. The evidence suggests they can generalize beyond their training in meaningful ways, even if imperfectly...if a human came out living but with a brain that had zero electrical activity, that would be extraordinary, we normally come out with a baseline of pre-programming. I sometimes think this debate happens because humans don't want to admit we're nothing more than LLMs programmed by nature and nurture, human seem to want to be especially special.
https://arxiv.org/abs/2206.07682
https://towardsdatascience.com/enhanced-large-language-model...
https://arxiv.org/abs/2308.00304
(and if MoRA is moving the goal posts, fine: RL/RT)
>if a human came out living but with a brain that had zero electrical activity, that would be extraordinary, we normally come out with a baseline of pre-programming.
That statement reveals deep deficiencies in your understanding of biological neural networks. "electrical activity" is very different from "pre-programming". Synapses fire all the time, no matter if meaningfully pre-programmed or not. In fact, electrical activity decreases over time in a human brain. So, if anything, programming over time reduces electrical activity (though there is no established causal link).
> I sometimes think this debate happens because humans don't want to admit we're nothing more than LLMs programmed by nature and nurture, human seem to want to be especially special.
It's not specific to humans. But indeed, we don't fully understand how brains of humans, apes, pigs, cats and other animals really work. We have some idea of synapses, but there is still a lot unclear. It's like thinking just because an internal combistion engine is made of atoms, and we mostly know how atom physics and chemistry work, that any body with this basic knowledge of atom physics can understand and even build an ICE. Good luck trying. It's similar with a brain. Yes, synapses play a role. But that doesn't mean a brain is "nothing more than an LLM".
1 reply →
A Person who memorizes something by rote, can pass many tests. From a test and verifiability perspective, they cannot be distinguished from someone who understands a subject.
An LLM can pass many tests, it is indistinguishable from someone who understands the subject.
Indistinguishable does not imply that the processes followed match what a human is doing when it understands a subject.
I use this when I think of humans learning - humans learn the most when they are playing. They try new things, explore ideas and build a mental model of what they are playing with.
To understand something, is to have a mental model of that thing in ones head.
LLMs have models of symbol frequency, and with their compute, are able to pass most tests, simply because they are able to produce chains of symbols that build on each other.
However, similar to rote learning, they are able to pass tests. Not understand. The war is over the utilitarian point “LLMs are capable of passing most tests”, and the factual point “LLMs dont actually understand anything”.
This articulation of the utilitarian point is better than the lazier version which says “LLMs understand”, and this ends up anthropomorphizing a tool, and creating incorrect intuitions of how LLMs work, amongst other citizens and users.