Comment by hasley
18 hours ago
There is a chance that someone from a completely different field came up with a solution for a tiny part of your problem.
If you can remember the content of any scientific publication and any book in the world, you are able to make use of this knowledge in every step of you proof.
However, this does now answer how the model came up with the specific route it has taken for the proof.
LLMs don't have super memory like that. I mean I don't know what this internal OAI model is, but at least for other LLMs, they aren't databases of training data with a smart search on top.
The agents here very likely used search. On top of that, they have boundless patience and can quickly process top K hits to find what they need. This is exactly the skill that is super useful for finding various niche sub-proofs that can help you build the final proof. A human mathematician is not going to digest 1000 papers from a different sub-field to find the needle they want, not knowing if it is actually there. AI can do it in few hours.
As Terry Tao said, LLMs are not outsmarting us, they are out remembering us.
I'm fairly sure your understanding is not fully accurate.
I'm not convinced anyone really understands the difference.
I did not mean to say that an LLM knows literally all the publications. But the abstract knowledge is probably encoded in the weights.
No but they have training data which teaches them certain amount of complex understandings and just not math but also physics. So this is one huge advantage.
And then they are for sure able to fill their context based on 'smart search on top' to actually progress further.