Comment by thomassmith65
14 days ago
Chomsky's problem here has nothing to do with his politics, but unfortunately a lot to do with his long-held position in the Nature/Nurture debate - a position that is undermined by the ability of LLMs to learn language without hardcoded grammatical rules:
Chomsky introduced his theory of language acquisition, according to which children have an inborn quality of being biologically encoded with a universal grammar
https://psychologywriting.com/skinner-and-chomsky-on-nature-...
I don't see how the two things are related. Whether acquisition of human language is nature or nurture - it is still learning of some sort.
Yes, maybe we can reproduce that learning process in LLMs, but that doesn't mean the LLMs imitate only the nurture part (might as well be just finetuning), and not the nature part.
An airplane is not an explanation for a bird's flight.
The great breakthrough in AI turned out to be LLMs.
Nature, for an LLM, is its design: graph, starting weights, etc.
Environment, for an LLM, is what happens during training.
LLMs are capable of learning grammar entirely from their environment, which suggests that infants are too, which is bad for Chomsky's position that the basics of grammar are baked into human DNA.
LLMs require vastly more data than humans and still struggle with some more esoteric grammatical rules like parasitic gaps. The fact grammar can be approximated given trillions of words doesn't explain how babies learn language from a much more modest dataset.
10 replies →