It still can't learn. It would need to create content, experiment with it, make observations, then re-train its model on that observation, and repeat that indefinitely at full speed. That won't work on a timescale useful to a human. Reinforcement learning, on the other hand, can do that, on a human timescale. But you can't make money quickly from it. So we're hyper-tweaking LLMs to make them more useful faster, in the hopes that that will make us more money. Which it does. But it doesn't make you an AGI.
That's not learning, though. That's just taking new information and stacking it on top of the trained model. And that new information consumes space in the context window. So sure, it can "learn" a limited number of things, but once you wipe context, that new information is gone. You can keep loading that "memory" back in, but before too long you'll have too little context left to do anything useful.
That kind of capability is not going to lead to AGI, not even close.
It still can't learn. It would need to create content, experiment with it, make observations, then re-train its model on that observation, and repeat that indefinitely at full speed. That won't work on a timescale useful to a human. Reinforcement learning, on the other hand, can do that, on a human timescale. But you can't make money quickly from it. So we're hyper-tweaking LLMs to make them more useful faster, in the hopes that that will make us more money. Which it does. But it doesn't make you an AGI.
It can learn. When my agents makes mistake they update their memories and will avoid making the same mistakes in the future.
>Reinforcement learning, on the other hand, can do that, on a human timescale. But you can't make money quickly from it.
Tools like Claude Code and Codex have used RL to train the model how to use the harness and make a ton of money.
That's not learning, though. That's just taking new information and stacking it on top of the trained model. And that new information consumes space in the context window. So sure, it can "learn" a limited number of things, but once you wipe context, that new information is gone. You can keep loading that "memory" back in, but before too long you'll have too little context left to do anything useful.
That kind of capability is not going to lead to AGI, not even close.
4 replies →
That’s not learning. That’s carrying over context that you are trusting is correctly summarised over from one conversation to the next.
3 replies →
> they update their memories
Their contexts, not their memories. An LLM context is like 100k tokens. That's a fruit fly, not AGI.
2 replies →