Comment by Mordisquitos
3 years ago
Note that Maciej a.k.a idlewords says (emphasis mine):
> The question hinges on whether LLM-like AI's are capable of recursive self-improvement
...but the evidence you suggest is:
> We already routinely train and finetune LLMs using text generated by other LLMs [...]
But there is still a huge gap between "self" improvement and improvements done that "we" trigger.
Now I do concede that you mention the next step being to close the feedback loop by replacing the humans doing the finetuning with another AI model doing so, but that is something that would open a whole new can of worms. For the researchers are improving LLMs with the input from other LLMs, sure... but why? Because of intentionality. And how do they evaluate the quality of the results? By their expectations as humans, in the context of their human culture and with their sensory experience of reality.
For an LLM to self-improve not only would it need to develop the self intention to do so (why develop it? which motivation?), but it would also need the ability to evaluate improvement (what is it "to improve"? how does it measure or sense it?).
Ultimately, without human- or real-world interaction, and without intrinsic motivation, a "self-improving" AI model would most likely result in something intelligent in a sense that is barely cogent for us, not because it is superior or inferior, but simply because nothing in it makes sense to our own purposes—harmless gibberish, as we humans would also be to the resulting self-improved AI.
Let us not forget that our own motivations as individual living creatures, as populations, and as cultures has been evolved over billions of years of natural selection which then framed millions of years of behavioural traits and tens of thousands of cultural evolution. Until AI can freely interact with the physical world and perform self-sustaining replication with the possibility of inheritable mutations, the only superintelligent AI that I would worry about would be that which is still fully in human hands.
> Note that Maciej a.k.a idlewords says...
That's why I added: "The obvious next step is to close the feedback loop with LLM-based agents instead of AI researchers/developers." We have early evidence that doing some like that might work, but no one knows for sure.
Yes, you did, and that's why I elaborated on why "closing the feedback loop" is barely enough to reach anything close to self-improvement. That is because self- requires intention to work in the direction of a particular goal, and -improvement requires an ability to evaluate whether the results are in line with it.
Going down to specifics, without the human intention of "getting good responses to human language prompts", and without the human ability to decide "this response was good" there is not much for an LLM to work on by itself.