Comment by bar8uyr

Ok my original comment was snarky (apologies to all three mods), but you've raised one of my favorite topics so I can't resist engaging a bit more seriously! In the second clause, that is -- the first is trivially true in some senses and trivially false in others, so it doesn't seem very productive to nail down.

So Chomsky thought of himself as a philosopher of science more than anything, and his life mission is/was to answer a fascinating question: what kind of creatures are we? There's been a growing consensus going back to at least Darwin that we're separated from the rest of Animalia by our "reason" (!= intuition != understanding != judgement != volition), but he helped popularize an old theory[1] that puts it kind of beautifully: human language is only possible in beings capable of producing an infinite set of meaningful outputs in response to a finite input.[2] And so far, that list includes only two definite members: Homo sapiens sapiens, and computer programs built upon large language models.[3]

So all that to say: yes, I think you're wrong; the programmatic ensembles that brought us today's breakthroughs are an "infinitely deep well". As long as we have questions to ask, they are not theoretically constrained from answering them.

Of course, this invites some obvious follow-ups on practicality; even the most impassioned ASI boomer wouldn't argue that the exact ensembles used here could answer every question in the universe. But A) that's possible because some infinities are bigger than others, and B) the exact same caveat applies to humans, whose cognitive capabilities are thoroughly determined by accidents of biology, culture, and episteme. The latter is the thrust of Chomsky's famous debate with Foucault, which is absolutely worth a watch for anyone who hasn't seen it (starting around ~18:00, AFAIR): https://youtu.be/3wfNl2L0Gf8?t=1080

---

TL;DR: I ultimately disagree on both points, yes. In the ways that matter, machines are no more constrained than animals.

[1]: https://en.wikipedia.org/wiki/Wilhelm_von_Humboldt

[2]: Many animals are capable of communication, often quite complex and fascinating communication at that; even flora and fungi communicate! But language is qualitatively distinct. You may have heard this in the pithy-yet-mindbending sentiment "we've taught many apes to communicate with sign language, but none of them have ever asked a question."

[3]: It's no surprise in hindsight that trying to build better autocorrect got us to AGI; I find it unspeakably tragic that Chomsky never got the chance to see how thoroughly he's been vindicated, in this regard. He had his stroke mere months into the singularity, and had a quite negative view of the approach before then.