Comment by krackers

Whatever idea yann has of JEPA and its supposed superiority compared to LLMs, he doesn't seem to have done a good job of "selling it" without resorting to strawmanning LLMs. From what little I gathered (which may be wrong), his objection to LLMs is something like the "predict next token" inductive bias is too weak for models to be able to meaningfully learn models of things like physics, sufficient to properly predict motion and do well on physical reasoning tasks.