Comment by computerex

You should look at some literature around it. I don't have time to pull it up now but it's been shown that much smaller small million parameters JEPA model outperforms much bigger LLMS in some applications. Keep in mind JEPA is area of active research.