Comment by CamperBob2
16 hours ago
The models, not the context. When it comes to weights, "quantity has a quality all its own" doesn't even begin to describe what happens.
Once you hit a billion or so parameters, rocks suddenly start to think.
16 hours ago
The models, not the context. When it comes to weights, "quantity has a quality all its own" doesn't even begin to describe what happens.
Once you hit a billion or so parameters, rocks suddenly start to think.
We're talking about context here though. The first couple seconds of Genie are great, but over time it degrades. It will always degrade, because it's hallucinating a world and needs to keep track of too many things.