Comment by jazzyjackson
19 hours ago
Yes the exponential self improvement folks have never heard of an eigenvalue I guess. You can loop forever using output as input but at some point the result will stop changing (depending on the function)
19 hours ago
Yes the exponential self improvement folks have never heard of an eigenvalue I guess. You can loop forever using output as input but at some point the result will stop changing (depending on the function)
I think the limit of what can be achieved with RL and synthetic data generation is better simply described as a leveling off of gains as you extract all the intelligence and knowledge from the original human training data.
Of course things will change at some point in the future as we go beyond LLMs, to build creative intelligence not just imitative/predictive intelligence, but right now these companies are stuck in this loop of building synthetic data and RLVR training from that, which means they are essentially building the "generative closure" of the original human training data - trying to squeeze all the juice out of it.
To go beyond this they need to add creativity of some sort to generate data that is not ultimately based on the original human training data. They could try something like brute force search (cf agent swarms/graphs), but this is just a more thorough way of exploring the search space defined by the training data - it may find you the "move 37" or low-hanging mathematical proof, but as Demis Hassabis has said, the goal of AGI is not to find move 37 but rather to create something capable of inventing as compelling a game as Go in the first place.
The name you are looking for is "fixed points", not "eingevalues".
that's not really how eigenvalues work... they specifically also model the case where the result keeps changing exponentially.
The claim is that the RSI operation is just finding a fixed point of improvement,
RSI(LLM) = RSI(LLM) -- for an optimal LLM* which is a fixed point of RSI
As for eigenvalues/vectors, they're fixed points of (1/val)A or A*val
Eigenvectors represent fixed directions, not fixed magnitudes. From Wikipedia:
> More precisely, an eigenvector v of a linear transformation T is scaled by a constant factor lambda when the linear transformation is applied to it: Tv = lambda v .
In other words, repeated multiplication of an eigenvector by a matrix can still create exponential growth.
>AI "recursively" improving itself in some exponential fashion until there is a bright flash of white light
Sounds like repetitive stress to me.
>loop forever using output as input but at some point the result will stop changing
Running in place will eventually wear you out too. Plus with some things it can be difficult to know for sure if that's where you are at the time.
Even worse may be if you were almost running in place, it could be orders of magnitude more difficult to discern, especially if the scale was massive to an unprecedented degree.