Comment by coderenegade

I'm out of the scene these days, and my field is more RL than LLMs, but my take is that all of this is telling us that meaning (and intelligence) is in the medium. If an LLM is a function mapping an input to some stateful internal representation that in-turn then gets mapped to an output, a smaller model probably doesn't have the capacity to learn that internal mapping, at least not from scratch. Or it would take a very long time, in same way that a naive sorting algorithm would still get there, but it might be computationally prohibitive in practice.

The bigger model has a better chance of gaining a foothold in that internal representation space where inputs are mapped to meanings and outputs, and eventually, it optimizes to the point where most of the weights aren't doing much. It's not immediately clear how much expressivity is required by the network to learn that space, but so far the answer seems to be in the billions of parameters.

The more interesting question to me is to what extent we should expect the models to be invariant to data. For instance, if I learn a certain type of analysis, that skill shouldn't depend on the data I'm looking at-- it should be repeatable for any given data of the same type/class. I'm curious to what extent skills are embedded in the weights versus data and "facts". My hunch for why mathematical reasoning and programming resulted in large step changes in model performance across the board is because these are inherently skills that are widely repeatable for a large class of tasks. The ability to express programmatic logic is invariant to both the language and the task at hand. And to me, that's how you get to smaller models: by focusing on the skills.