Comment by antics

3 days ago

If you'd like to use that definition of "predictive" in this discussion that's 100% fine with me. But, you should do so knowing that in ML and AI "prediction" is a term of art that means "a statistical model's output given some input." LLMs are one example, in that they take as input a sequence of tokens and predict the next token.

In this specific context, the point is that LLMs and MT "predict" lucid-sounding sentences that incorporate linguistic structure (syntactic and semantic) that are well beyond the predictive abilities of the Chomskyan theory of syntax (which is itself a generative model). In other words, the Chomskyan model of syntax is descriptive of much of the rich variation in real-world human language. But it is not predictive in the sense that it does not provide a constructive algorithm to emit lucid-sounding sentences (and does not attempt to, either).