Comment by nowittyusername
7 months ago
This is an important moment. We now have verifiable evidence that these systems can do new useful research that has actual value in the real world. That 1% savings is only the start as well. I would expect the compounding number of gains to be significant over some time. Also in a way this process was used to make gemini 2.5 pro better, so its like a baby step towards recursive self improvement. Not fully automated yet, but there are hints of where this is going.
Genetic programming systems have periodically made improvements to algorithms (dating back decades). Whether LLM-powered GP, which is effectively what this is, will be a step change or an evolution of that is still an open question I think. I'm also a little wary of reading too much into the recursive self-improvement idea, because "the GP system can use GP to improve the GP system itself!" is a very old idea that just has never worked, although I realize that isn't proof that it won't eventually work.
Some related work from a different company: https://sakana.ai/ai-cuda-engineer/
And some academic papers kind of in this space: https://arxiv.org/abs/2206.08896, https://arxiv.org/abs/2302.12170, https://arxiv.org/abs/2401.07102
It's always "revolutionizing our internal workflows" or "30% of code at Microsoft is AI now" but never improving a codebase you can actually see
Making a significant improvement to the state of the art of one particular algorithm is one thing, but I've seen new tools do that since the 80s
I'll be convinced when LLMs start making valuable pull requests, non-obvious corner cases or non-trivial bugs in mature FOSS projects
Is it new? I'm getting mixed messages from the posts here. On one side there is evidence that 48 and 46 multiplication solutions have been known (and could have found themselves in the model training data).
On the other side I see excitement that the singularity is here.
If the latter were the case surely we wouldn't be reading about it in a published paper, we would already know.
Let's assume that the 46 multiplication algorithm was known, prior to AlphaEvolve re-discovering it. AlphaEvolve still has made an improvement to a performance critical area that has had likely had thousands of engineer-hours put into it. None of those engineers apparently knew about the improved algorithm, or were able to implement the algorithm. This is empirical evidence of an LLM outperforming its (domain expert) human counterparts.
Isn't this like comparing a human historian to Wikipedia though? Of course the knowledge in Wikipedia will in most cases beat the human. However, that's not the kind of thing we're looking for here.
2 replies →