Comment by bloomingkales
3 months ago
If an LLM output is like a sculpture, then we have to sculpt it. I never did sculpting, but I do know they first get the clay spinning on a plate.
Whatever you want to call this “reasoning” step, ultimately it really is just throwing the model into a game loop. We want to interact with it on each tick (spin the clay), and sculpt every second until it looks right.
You will need to loop against an LLM to do just about anything and everything, forever - this is the default workflow.
Those who think we will quell our thirst for compute have another thing coming, we’re going to be insatiable with how much LLM brute force looping we will do.
I can't believe this hasn't been done yet, perhaps it is a cost issue.
My literal first thought about AI was wondering why we couldn't just put it in a loop. Heck, one update per day, or one update per hour would even be a start. You have a running "context", the output is the next context (or a set of transformations on a context that is a bit larger than the output window). Then ramp that up ... one loop per minute, one per second, millisecond, microsecond.
The hard part is coming up with a good way to grade results. Which you need to update the weights based on the outcome, otherwise the model will not actually learn anything.
For the "looping" I'm talking about you don't need to update the weights. It is simply, old context in, new context out, new context in, new-new context out, etc.
Of course, keeping that coherent over numerous loops isn't going to be easy. No doubt there is a chance it goes off the rails. So you might have a section of context that is constantly stable, a section of context that updates each loop, etc.
In the other response to my comment someone mentioned eventually updating the weights (e.g. daily) and you would in that case have to have some kind of loss function.
4 replies →
Same. And the next step is that it must feed back into training, to form long-term memory and to continually learn.
I analogize this with sleep. Perhaps that is what is needed, 6 hours offline per day to LoRa the base model on some accumulated context from the day.
1 reply →
This is a fantastic insight and really has my gears spinning.
We need to cluster the AI's insights on a spatial grid hash, give it a minimap with the ability to zoom in and out, and give it the agency to try and find its way to an answer and build up confidence and tests for that answer.
coarse -> fine, refine, test, loop.
Maybe a parallel model that handles the visualization stuff. I imagine its training would look more like computer vision. Mind palace generation.
If you're stuck or your confidence is low, wander the palace and see what questions bubble up.
Bringing my current context back through the web is how I think deeply about things. The context has the authority to reorder the web if it's "epiphany grade".
I wonder if the final epiphany at the end of what we're creating is closer to "compassion for self and others" or "eat everything."
> If an LLM output is like a sculpture, then we have to sculpt it. I never did sculpting, but I do know they first get the clay spinning on a plate.
That’s pottery, not sculpture. Traditionally in sculpture you start from a block of marble or wood, but you can also make sculptures of cast bronze or welded steel (or clay, but you don’t use a spinning plate).
Thank you for the clarification. I wanted to use some kind of visual to show the model in a loop. Otherwise, I’d just have to say explicitly that the sculptor is the one in the loop, as in the person will not stop chiseling. It’s in this infinite chiseling that we get our answers (same thing as finding a limit in calculus as it approaches infinity, we will never get the discrete answer, but we will get infinitely close enough to label a discrete point confidently).
In other words, we fly as close to the sun as possible and get our measurements :)