Comment by zoogeny
2 months ago
I can't believe this hasn't been done yet, perhaps it is a cost issue.
My literal first thought about AI was wondering why we couldn't just put it in a loop. Heck, one update per day, or one update per hour would even be a start. You have a running "context", the output is the next context (or a set of transformations on a context that is a bit larger than the output window). Then ramp that up ... one loop per minute, one per second, millisecond, microsecond.
The hard part is coming up with a good way to grade results. Which you need to update the weights based on the outcome, otherwise the model will not actually learn anything.
For the "looping" I'm talking about you don't need to update the weights. It is simply, old context in, new context out, new context in, new-new context out, etc.
Of course, keeping that coherent over numerous loops isn't going to be easy. No doubt there is a chance it goes off the rails. So you might have a section of context that is constantly stable, a section of context that updates each loop, etc.
In the other response to my comment someone mentioned eventually updating the weights (e.g. daily) and you would in that case have to have some kind of loss function.
Then I'm not quite sure what benefit you expect to derive from it? Making e.g. QwQ-32 loop isn't hard - it often does it all by itself, even. But it doesn't translate to improvements on every iteration; it just goes in circles.
1 reply →
When you look out your eyes, that rectangular viewport is all your context. For example, we cannot fit Jupiter into your viewport.
So, if we can never fit large concepts like Jupiter into your viewport (your context window), does it follow that you will never understand Jupiter? Or is there no way to take a picture of Jupiter and make it smaller so it fits into your viewport?
See, coherence is just resizing or reimagining things so it fits into context. Context can never hold it all because we have the capacity to always imagine infinitely large things.
So I don’t fully know if it’s old context out, new context in, but could be just the right context, just enough of it, and just the right looking context so we can assemble the right worldview. This process would have to go on infinitely, and that is the world as you see and understand it.
Same. And the next step is that it must feed back into training, to form long-term memory and to continually learn.
I analogize this with sleep. Perhaps that is what is needed, 6 hours offline per day to LoRa the base model on some accumulated context from the day.
LLMs need to sleep too. Do they dream of electric sheep?