Comment by loosetypes
1 day ago
Mind linking any examples (or categories) of problems that are definitively not in pre training data but can still be solved by LLMs? Preferably something factual rather than creative, genuinely curious.
Dumb question but anything like this that’s written about on the internet will ultimately end up as training fodder, no?
How about the International Math Olympiad?
https://arstechnica.com/ai/2025/07/google-deepmind-earns-gol...
You're saying they don't use math textbooks and math forums to train LLMs, then?
The problems are not in textbooks. I’m curious what would count as an out of distribution problem for you. Only problems no one knows how to solve?
You can apply this same argument to humans, 99.999% of people will not be able to escape it.
In the case of the Math Olympiad, the students who take it grind hours a day for months on practice problems and past Olympiad problems.