Comment by gametorch
5 days ago
Getting gold at the IMO is pretty damn hard.
I grew up in a relatively underserved rural city. I skipped multiple grades in math, completed the first two years of college math classes while in high school, and won the award for being the best at math out of everyone in my school.
I've met and worked with a few IMO gold medalists. Even though I was used to scoring in the 99th percentile on all my tests, it felt like these people were simply in another league above me.
I'm not trying to toot my own horn. I'm definitely not that smart. But it's just ridiculous to shoot down the capabilities of these models at this point.
The trouble is, getting an IMO gold medal is much easier (by frequency) than being the #1 Go player in the world, which was achieved by AI 10 years ago. I'm not sure it's enough to just gesture at the task; drilling down into precisely how it was achieved feels important.
(Not to take away from the result, which I'm really impressed by!)
The "AI" that won Go was Monte Carlo tree search on a neural net "memory" of the outcome of millions of previous games; this is a LLM solving open ended problems. The tasks are hardly even comparable.
A "reasoning LLM" might not be conceptually far from MCTS.
I really don't like the use of the word memory here, even in quotes. AlphaGo has a much better "understanding" of Go positions than mine (7k).
And then they created AlphaGo Zero, which is not trained on any previous games, and it was even stronger!
https://deepmind.google/discover/blog/alphago-zero-starting-...
1 reply →
Nothing that uses a mathematical model for solving a problem will ever reason because reasoning can only be done by things we don't understand...