Comment by PaulDavisThe1st
14 days ago
Alternatively, what Chomsky was thinking about with his universal grammar idea is something implicitly present in both our minds and an LLM i.e. "it's the wiring, stupid".
I'm not sure there's much evidence for this one way or another at this point.
>is something implicitly present in both our minds and an LLM
The LLM doesn't start with any real structure besides the network of ops though. If there is any induced structure, it's learnable from the data. And given enough data the base network is sufficient to learn the "grammar" of not just human language but more complex CFGs and things you wouldn't traditionally consider "languages" as well (e.g. audio, images). In a sort of chicken/egg scenario, the morasses of data gives rise to the structures needed to parse and generate that data.