Comment by dymk
7 months ago
Let's assume that the 46 multiplication algorithm was known, prior to AlphaEvolve re-discovering it. AlphaEvolve still has made an improvement to a performance critical area that has had likely had thousands of engineer-hours put into it. None of those engineers apparently knew about the improved algorithm, or were able to implement the algorithm. This is empirical evidence of an LLM outperforming its (domain expert) human counterparts.
Isn't this like comparing a human historian to Wikipedia though? Of course the knowledge in Wikipedia will in most cases beat the human. However, that's not the kind of thing we're looking for here.
I don't think that's quite the comparison we're looking for though, because this wasn't just rote data retrieval. It was the recognition of patterns that humans could have noticed given enough time, but had not. It's more like a system that can make inferences as insightful as a thousand skilled human historians typing away at typewriters, armed with the collective knowledge of Wikipedia, in a short period of time.