Comment by f30e3dfed1c9

I think that misses the point entirely. Even if you constructed some system the output of which could not be distinguished from human-produced language but that either (1) clearly operated according to principles other than those that govern human language or (2) operated according to principles that its creators could not adequately explain, it would not be of that much interest to him.

He wants to understand how human language works. If I get him right — and I'm absolutely sure that I don't in important ways — then LLMs are not that interesting because both (1) and (2) above are true of them.