Comment by computerex
2 days ago
You heard of JEPA? LLM's have all sorts of garbage they have memorized. Reasoning in latent space instead of in text significantly reduces the number of needed parameters.
2 days ago
You heard of JEPA? LLM's have all sorts of garbage they have memorized. Reasoning in latent space instead of in text significantly reduces the number of needed parameters.
JEPA is a joke, let me know when those models do anything useful.
You should look at some literature around it. I don't have time to pull it up now but it's been shown that much smaller small million parameters JEPA model outperforms much bigger LLMS in some applications. Keep in mind JEPA is area of active research.