Comment by bamboozled
18 hours ago
Why would the trick have any "origins", isn't this model creating new techniques never before seen or imagined?
18 hours ago
Why would the trick have any "origins", isn't this model creating new techniques never before seen or imagined?
There is a chance that someone from a completely different field came up with a solution for a tiny part of your problem.
If you can remember the content of any scientific publication and any book in the world, you are able to make use of this knowledge in every step of you proof.
However, this does now answer how the model came up with the specific route it has taken for the proof.
LLMs don't have super memory like that. I mean I don't know what this internal OAI model is, but at least for other LLMs, they aren't databases of training data with a smart search on top.
The agents here very likely used search. On top of that, they have boundless patience and can quickly process top K hits to find what they need. This is exactly the skill that is super useful for finding various niche sub-proofs that can help you build the final proof. A human mathematician is not going to digest 1000 papers from a different sub-field to find the needle they want, not knowing if it is actually there. AI can do it in few hours.
As Terry Tao said, LLMs are not outsmarting us, they are out remembering us.
I'm fairly sure your understanding is not fully accurate.
1 reply →
I did not mean to say that an LLM knows literally all the publications. But the abstract knowledge is probably encoded in the weights.
No but they have training data which teaches them certain amount of complex understandings and just not math but also physics. So this is one huge advantage.
And then they are for sure able to fill their context based on 'smart search on top' to actually progress further.
As I understand it it's undetermined yet whether LLMs can actually come up with anything novel or are instead pulling from their incredibly deep corpus of knowledge to present solutions that were there but we didn't realize it because our brains aren't libraries of almost all human writing.
Synthetic data allows them to train well past the limits of human writing.
What's an example of synthetic data?
Only in the same sense it's not yet determined about humans, either.
Not so sure. Was everything already "there" before humans existed?
10 replies →
No this is not an issue. As long as their is a way of verifying things, they do the same thing with creating novel things as humans: Searching through an infinite space of possibilities opitmized by knowledge.
They combine things, verify it and if it works and progresses the problem, they created something new.
Let me introduce you to 'obscure Russian mathematicians'.