Comment by HarHarVeryFunny
18 hours ago
RSI is a fetishistic term among the singularity crowd, who imagine AI "recursively" improving itself in some exponential fashion until there is a bright flash of white light and it reveals itself in the form of god. Or something like that.
I don't know why whoever coined the term chose "recursive" rather than "iterative" - just sounds more likely to lead to infinite regress I suppose.
This notion of recursive/iterative self-improvement, whereby generation #1 AI improves itself to create generation #2, then generation #2 further improves itself to create generation #3, etc, seems to conflict with the reality that what we have with LLMs is models whose performance/capability is defined by data, not code, so the most you can do is have your LLM design synthetic data, or just do Karpathy-style "auto research" where all you are doing is using the LLM to automate your experiments.
At the end of the day, each experiment, designed by a person and/or LLM, then needs to compete with all your other ideas for compute to be tested at scale, and no amount of recursion or self-improvement will materialize an infinite amount of compute out of thin air, so your recursively synthetic-data gobbling LLM will continue to improve at the same pace it ever did.
“Recursive” is a reasonable term because the generation N AIs will train the Generation N+1 AIs. The term “iterative” doesn’t reflect this nuance as well IMO.
Recursion reduces each step toward a base case: each step is defined in terms of previous/simpler steps, not more advanced ones. The "recursive" in "recursive self improvement" has things precisely backward. Iteration correctly describes a process where each step is the starting point of its successive step, so it should be "iterative self improvement" but I guess that didn't sound as cool.
I think you’re conflating the direction of definition with the direction of evaluation.
Compare the similarity of:
With:
The latter is a classic example of recursion. So why isn’t the former?
Edit: formatting
1 reply →
It's not a nuance, it's a sequence.
I felt like the scaling laws were magical thinking, but apparently they work. However I still do not understand why we should expect exponential improvements due to this automated process. My intuition is that the first iteration of it should result in a noticeable capability increase (though I think these labs were already using a lot of AI to orchestrate training the current model anyway), and then the second iteration of it should be nearly identical in capability to the first, unless more data is involved, more compute is involved, or the model is bigger.
You can get exponential growth from completely ordinary feedback loops. You start with some amount of stuff, you do a series of steps and you end up with more of the same stuff you started with. As you keep going through the loop, the stuff you have grows exponentially. That's for example how exponential economic growth works.
Of course data, compute and model size are not held constant. You start with some money and use it to acquire researchers, data and compute, and have the researchers produce a big model and you use that model to get more money, and you use the additional money for more researchers, more data, and more compute to produce a bigger model. This is what has propelled exponential AI progress so far.
Recursive self-improvement is invoked to predict superexponential growth. The idea is that instead of only using the model to make more money, you add it to the researchers to speed up the loop, so not only is the money growing with every iteration, the iteration time also gets shorter, producing growth that is faster than exponential.
The problem with this simplistic prediction is that it assumes additive and multiplicative relationships of the form money = (researchers + AI)×compute_spend, but if doing more research paid off so reliably, you could also just hire more researchers, abstractly money = research_spend×compute_spend and with a balanced allocation of research and compute, you would get a money-squaring machine even without using AI for AI research.
And the reason this doesn't work in reality is that there are diminishing returns everywhere. You can also see this in the OpenAI post, where they write 7 times as much code to run 1.6 times as many experiments, and those additional experiments probably only result in minor improvements to model quality.
AI can compress AI nearly losslessly.
Yes the exponential self improvement folks have never heard of an eigenvalue I guess. You can loop forever using output as input but at some point the result will stop changing (depending on the function)
The name you are looking for is "fixed points", not "eingevalues".
that's not really how eigenvalues work... they specifically also model the case where the result keeps changing exponentially.
The claim is that the RSI operation is just finding a fixed point of improvement,
RSI(LLM) = RSI(LLM) -- for an optimal LLM* which is a fixed point of RSI
As for eigenvalues/vectors, they're fixed points of (1/val)A or A*val
1 reply →
>AI "recursively" improving itself in some exponential fashion until there is a bright flash of white light
Sounds like repetitive stress to me.
>loop forever using output as input but at some point the result will stop changing
Running in place will eventually wear you out too. Plus with some things it can be difficult to know for sure if that's where you are at the time.
Even worse may be if you were almost running in place, it could be orders of magnitude more difficult to discern, especially if the scale was massive to an unprecedented degree.
What will prevent LLMs from designing robot control circuitry and participating in increase of chip production/design and physical experimentation?
How do you think why there's this fad of producing general purpose humanoid robots?
> How do you think why there's this fad of producing general purpose humanoid robots?
For doing physical work?
So a swarm of robots builds the shell of your fab overnight, and then what? Where is the EUV machine coming from?
So far the most we're seen TeslaBot do is serve drinks via tele-operation, and I don't think it's exactly built for construction site work.
For example, TSMC uses behavioral cloning to scale up human-bottlenecked parts of the manufacturing process to meet the growing demand, while automated research laboratories do thousands experiments in parallel to find better manufacturing processes.
> What will prevent LLMs from designing robot control circuitry and participating in increase of chip production/design and physical experimentation?
Money, regulations, EUV machine lead-times, global helium supply, reality ...
It's funny that we've got the Dwarkesh contingent saying that GPUs will become infinitely expensive, and now another contingent saying that they will become infinitely abundant.
Even if compute were free, and/or the AI was so smart that it picked the right experiments to run every time ("make no mistakes"), you still have to actually train the model, which takes months, and if model Ver. N+1 depends on model Ver. N, then it's iterative regardless of how much compute you have.
Who's saying that compute will become infinitely abundant? "Singularity" is just a way of saying that known models begin to give absurd predictions. Anyway, intelligence is a way of overcoming obstacles. 10 million tonnes of helium is a nice head start and retraining models from scratch is not guaranteed to last forever.
2 replies →