Comment by mdp2021
10 days ago
I have not followed all of LeCun's past statements, but -
if the "core belief" is that the LLM architecture cannot be the way to AGI, that is more of an "educated bet", which does not get falsified when LLMs improve but still suggest their initial faults. If seeing that LLMs seem constrained in the "reactive system" as opposed to a sought "deliberative system" (or others would say "intuitive" vs "procedural" etc.) was an implicit part of the original "core belief", then it still stands in spite of other improvements.
If you say LLMs are a dead end, and you give a few examples of things they will never be able to do, and a few months later they do it, and you just respond by stating that sure they can do that but they're still a dead end and won't be able to do this.
Rinse and repeat.
After a while you question whether LLMs are actually a dead end
This is a normal routine topical in Epistemology in the perspective of Lakatos.
As I said, it will depend on whether the examples in question were actually substantial part of the "core belief".
For example: "But can they perform procedures?" // "Look at that now" // "But can they do it structurally? Consistently? Reliably?" // "Look at that now" // "But is that reasoning integrated or external?" // "Look at that now" // "But is their reasoning fully procedurally vetted?" (etc.)
I.e.: is the "progress" (which would be the "anomaly" in scientific prediction) part of the "substance" or part of the "form"?