Comment by tptacek
1 day ago
I think you can reasonably assume that frontier models are using SymPy or something like it any time interesting math gets into the picture, and the person driving Fable here is an accomplished mathematician, but I don't think we can reasonably assume either extensive prompting or brute-force compute in any sense other than what it normally takes Fable to, say, whip up a calculator app.
Fable "whipping up" SymPy like a "calculator app" is one such interaction that seems very plausible (in addition to SymPy providing feedback when training models). The scale of compute available to an Anthropic employee for such SymPy calls when using Fable is likely one of multiple factors for this counterexample being found in 2026. Unfortunately, we aren't going to be able to really know the various factors that best explain why an Anthropic employee was able to announce a counterexample this past weekend. For all we know, the counterexample was found by Anthropic employees months ago and used for training the Fable model used this past weekend.
I love this. Anthropic employees just randomly have solutions to Smale's open problems in their back pockets, waiting for the right moment to sprinkle them into the training set.
The two big discoveries both came from the negligible handful of mathematicians working at OpenAI/Anthropic in spite of many orders of magnitude more mathematicians using them outside of the companies. I don't see any way to explain this without assuming that the limiting factor is the ability to burn a few rainforests worth of tokens in pursuit of something publishable.
I think it would also explain their opacity towards the process. Being able to solve such well known problems in a nice replicable 1-2-3 way would be far more effective marketing than their complete opacity outside of the result, which suggests that they feel transparency is not in their best interest for some reason.
> The two big discoveries both came from the negligible handful of mathematicians working at OpenAI/Anthropic in spite of many orders of magnitude more mathematicians using them outside of the companies
Well, mathematicians not working for Anthropic/OpenAI are heavily disincentivised from reporting that their discoveries were made using AI. If e.g. the idea that resolved the Mahler conjecture came from AI, it's not like we'd ever know.
Would you ordinarily be able to "explain this", if a mathematician had come up with this on their own? How would that story go?
Bayesian probability. Were outcomes being driven by 'normal' usage of LLMs then it's extremely improbable that both big discoveries would come from the small number of people working at the companies. That suggests working at the companies is more the decisive factor than the LLMs in and of themselves.
And what do you get from working at the company? Likely a rather massive token/processing budget. The companies opacity towards the path to these discoveries also makes this further probable as 'spend millions of dollars in tokens' is a somewhat less attractive narrative than the implied narrative of 'just use Fable.'