Comment by drcode
3 months ago
An LLM just solved 10 top tier math problems this week. It seems very likely that in 6 months they will solve 100 top tier math problems, maybe 1 millenium math problem, maybe 10 top tier physics problems, etc.
It's only going to get crazier.
Math is verifiable. LLMs will continue to make strides in such search spaces where all they need to do is try->verify->repeat. You can't train an LLM on how to end the war in Iran or Ukraine, nor how to solve hunger, nor climate change, etc. Real issues.
And LLMs still suck at art and writing.
Three years ago, people were saying "LLMs just generate plausible-sounding text, they don't understand the notion of truth so they can't do verifiable work like math proofs."
They still can't. But very smart humans constructed ways to use the monkeys with typewriters (with a statistical advantage) to find correct answers to problems where they already knew how to verify the answer.
6 replies →
Right, which was true at the time. So hundreds of billions of dollars have been poured into making LLMs better at these tasks via pretraining, RL, RLHF, post training, etc. again all with something verifiable in the loop. In order to improve the thing in the loop, the loop itself needs to be verifiable.
There have only been a few thousand wars, and they’re all different and all different in the world in which they occurred. The dimensionality is absurd, which is not a problem for LLMs if there’s enough data, but in this case there isn’t.
> they don't understand the notion of truth so they can't do verifiable work like math proofs.
Nobody that understands automated proof checking was claiming that.
You understand that those are different kinds of "truth", right?
> they don't understand the notion of truth so they can't do verifiable work like math proofs."
a) No one ever said that.
b) Your comment shows a lack of understanding of the notion of truth.
Yeah but... Both things are true. LLMs are doing things we would have considered absolute AGI god tier magic 5 years ago. They also still can't count to 100 yet.
There are definitely LLMs that can count to 100
If you think all they need to do is "just that" (try, verify, repeat as you say) then I think you're very, very mistaken.
There is no way for the LLM to bruteforce the search space any better than a human. What it can do better, tho, is to make connections between seemingly (for us) unconnected notions and join them, then verify if that's right.
Your view is not only wrong but also condescending in this day and age.
That’s literally how AlphaEvolve works. And presumably how OpenAI’s research loop works if they’d ever publish it.
They put an LLM in a loop, with a reward function, and keep trying to get a higher score. The reward function for AlphaEvolve is “did this code get a better score or faster”, for math research it’s “did it write a LEAN proof”.
I’m open to hearing that I’m wrong but I don’t think I am. I agree that LLMs make connections that humans wouldn’t, but eventually you need to verify those because otherwise the connection it made may as well be a lie unless proven otherwise.
Claude had some pretty good ideas about how to end the war in Iran when I asked it a while ago (The US should sue for peace, I believe was the conclusion).
>An LLM just solved 10 top tier math problems this week
and 30 years ago a computer beat Gary Kasparov, if you'd listened to Hans Moravec you wouldn't be surprised that the first thing that gets automated is intellectual domain expertise.
Things are going to get crazy when it can figure out how to walk into a random house a and brew a cup of coffee, not do math
When we've done both we will have fully automated Erdős.
regular LLM use makes people sound like they’re on meth so we must be getting close!
Suppose LLM solved all math problems. So what? It’s not like diplomacy and war will end. Are humanity’s problems really constrained on our intellectual ability? If anything most evidence points to cultural failing and all AI will do is enable unprecedented oppression due to said skills at intellectual tasks.
As you read this some poor person is starving. Humanity already possesses the ability to identify said person and send them aid. How is AI going to help here?
The ultimate fallacy is that all technological progress will benefit mankind. That will be true until it is not.
All technological progress will benefit an increasingly smaller (and wealthier) slice of mankind.
I have no idea if diplomacy and/or war will end, I just hope we humans get to hang around a few more years
It's an interesting time when the bar has moved to "It’s not like diplomacy and war will end". But that is on the table. Nuclear weapons caused the end of wars as we know them. Great powers no longer directly attack each other. And AI is much more powerful than nuclear weapons, with an equal capacity for damage and a much greater capacity for good.
By the way, I donate monthly to GiveWell and the shrimp welfare project. Do you donate monthly to starving people? Most people don't, and the reason is simple: at the end of the day, most people just don't care that much about helping a starving person far away. They also don't care much about things like that, animals in factory farms, or earthquakes that kill hundreds of thousands. But ideally, we could find a way to empower people such that the minority who do care can make a big difference.
How is AI "more powerful than nuclear weapons" etc when they're two completely different things? Please explain to me how AI can glass the planet via mutually assured destruction.
1 reply →
Yikes
4 replies →
> Suppose LLM solved all math problems. So what? It’s not like diplomacy and war will end.
These are two vastly different statements, and the the war and diplomacy end is almost trivial by comparison, it would absolutely be solved before we solve all math in even the most steelmanned version.
https://dsimanek.vialattea.net/twain.htm
Has an LLM ever intentionally made you laugh? Cry?
Have you?
(I'm trolling, I'm not sure what you're asking or why)
Not OP but it implies all efforts are not intentional but at best by chance. For instance, the state of freely available LLM chatbots still makes me cry by the amount of crap they spew out confidently, which is not intentional. (tiny bit of sarcasm, but only a tiny bit)
Incoherent question because LLMs don't have intentions.
[dead]
How many maths problems does one have to solve before becoming god?