So many mathematicians over the years tried hard and failed, but now Anthropic just for some PR magically did it? And this after LLMs obtaining different math wins? What is your logic here really escapes my understanding.
The parent's absolutely nonsensical post highlights how polarized AI (as everything else) is today. I can understand someone being opposed to AI on moral, cost-benefit or productivity grounds. But we're seeing a lot of extremist "AI is good for nothing" posts out there nowadays.
I don't think it is nonsensical at all. The author and his collaborator both appear to be bright people, so there's a good chance they had to offer non-trivial insights to guide the LLM, yet it's clearly in the interest of his employer to downplay whatever personal contribution they provided.
Edit: Now the OP is flagged/dead for some reason. You could disagree on their take (calling it a marketing stunt is maybe a bit much), but I think the argument is sound, so flagging seems counterproductive to the discussion.
Those silly advertisers do everything for exposure and if that means digging yourself into a niche alleged mathematical theorem to refute it, it is what needs to be done!
Of course it would be really interesting how Claude approached this. Probably with some constraints regarding the input. And it would be interesting what these constraints were.
I mean we have no idea what happened exactly, how Fable was used, how many times it was run, whether earlier models were also tried, what was the prompt, how long it run for, etc etc. All we have to go by is a tweet.
What you're asking for is exactly the sort of thing that belongs in, and will appear in, a journal article. There will likely be a preprint on arxiv, so you might keep an eye out for that.
In any case, the fact that it was found by a commercial model means that the unfiltered reasoning trace isn't available even to the original author. So there are aspects of the problem-solving process we'll never see. Even if we did get access to the reasoning trace it wouldn't necessarily be definitive, given how these things work.
Hopefully it'll be possible to get the same solution from an open-weight model like one of the 3T heavyweights that are said to be coming up for release. If so, the chain of thought can be scrutinized in-depth.
They gave their "logic", such as it is ... and it's utterly irrational.
Note that the "they" who published the counterexample on X is some rando mathematician (Levent Alpöge) working for Anthropic, not Anthropic the organization. He posted the counterexample in a tweet -- reason enough for "not disclosing the LLM chat session". There's no reason to think that it won't provided if asked for, but it hardly seems relevant.
> There's no reason to think that it won't provided if asked for, but it hardly seems relevant.
My guess is that the chat will look similar to a full transcription of a (multi month?) discussion between a few mathematicians. Full of dead ends and stupid errors (bit by the human and Claude) that would be embarrassing. We all know how bad it is, and we prefer to keep it behind the curtain.
if you ask the chatbots for "list of top unsolved math problems", the JC comes in at a ranking of around #10 - #20. what, a problem that's been unsolved since 1939 was cracked because anthropic has an underground sweatshop of math Phds cranking out research, just so that they can slap "made by AI" on it? hell, maybe lizard people did it.
1. There is no such thing as "Artificial Intelligence" (but yes, machine learning and LLMs are real and powerful things)
2. Calling LLMs 'AI' is part of the grifting hype that has been wildly prevalent in the US corporate space. It's very natural and also very good that there is now backlash to and scepticism around all of the hype that is has been generated over the last 5 or so years
There will be more and more mathematical proofs provided by LLMs as they improve. If you assume every one is just a marketing tactic, you'll drive yourself insane.
So many mathematicians over the years tried hard and failed, but now Anthropic just for some PR magically did it? And this after LLMs obtaining different math wins? What is your logic here really escapes my understanding.
The parent's absolutely nonsensical post highlights how polarized AI (as everything else) is today. I can understand someone being opposed to AI on moral, cost-benefit or productivity grounds. But we're seeing a lot of extremist "AI is good for nothing" posts out there nowadays.
I don't think it is nonsensical at all. The author and his collaborator both appear to be bright people, so there's a good chance they had to offer non-trivial insights to guide the LLM, yet it's clearly in the interest of his employer to downplay whatever personal contribution they provided.
Edit: Now the OP is flagged/dead for some reason. You could disagree on their take (calling it a marketing stunt is maybe a bit much), but I think the argument is sound, so flagging seems counterproductive to the discussion.
6 replies →
If you can call it polarized when a majority of people are just happily using the technology while a minority keeps spreading delusions and hatred.
Those silly advertisers do everything for exposure and if that means digging yourself into a niche alleged mathematical theorem to refute it, it is what needs to be done!
Of course it would be really interesting how Claude approached this. Probably with some constraints regarding the input. And it would be interesting what these constraints were.
Author probably doesn't want to show the prompt because they are now trying to find a bunch of other counter-examples with the same prompt
1 reply →
I mean we have no idea what happened exactly, how Fable was used, how many times it was run, whether earlier models were also tried, what was the prompt, how long it run for, etc etc. All we have to go by is a tweet.
Why not be skeptical about that?
What you're asking for is exactly the sort of thing that belongs in, and will appear in, a journal article. There will likely be a preprint on arxiv, so you might keep an eye out for that.
In any case, the fact that it was found by a commercial model means that the unfiltered reasoning trace isn't available even to the original author. So there are aspects of the problem-solving process we'll never see. Even if we did get access to the reasoning trace it wouldn't necessarily be definitive, given how these things work.
Hopefully it'll be possible to get the same solution from an open-weight model like one of the 3T heavyweights that are said to be coming up for release. If so, the chain of thought can be scrutinized in-depth.
7 replies →
They gave their "logic", such as it is ... and it's utterly irrational.
Note that the "they" who published the counterexample on X is some rando mathematician (Levent Alpöge) working for Anthropic, not Anthropic the organization. He posted the counterexample in a tweet -- reason enough for "not disclosing the LLM chat session". There's no reason to think that it won't provided if asked for, but it hardly seems relevant.
> There's no reason to think that it won't provided if asked for, but it hardly seems relevant.
My guess is that the chat will look similar to a full transcription of a (multi month?) discussion between a few mathematicians. Full of dead ends and stupid errors (bit by the human and Claude) that would be embarrassing. We all know how bad it is, and we prefer to keep it behind the curtain.
1 reply →
A few months ago I asked a model how many primes are divisible by 35 with a remainder of 6. It confidently replied 'none'.
Counterexample: 35 + 6.
5 replies →
> some rando mathematician (Levent Alpöge) working for Anthropic, not Anthropic the organization
Why do you trust a random stranger so much? Will you hand over your car keys to a random stranger? Sharing the chat will take 30s of their time.
> There's no reason to think that it won't provided if asked for
But they didn't provide it.
15 replies →
[flagged]
Cheating how? The proof is in the pudding...
3 replies →
Dude what?
There's a big difference between one shotting a counterexample using AI and using AI to find a counter-example by brute-force.
Both are impressive, of course, but they're hardly comparable.
I'm not so sure. People had been trying to use brute force to find counterexamples before.
23 replies →
if you ask the chatbots for "list of top unsolved math problems", the JC comes in at a ranking of around #10 - #20. what, a problem that's been unsolved since 1939 was cracked because anthropic has an underground sweatshop of math Phds cranking out research, just so that they can slap "made by AI" on it? hell, maybe lizard people did it.
This anti-AI sentiment is getting borderline insane.
1. There is no such thing as "Artificial Intelligence" (but yes, machine learning and LLMs are real and powerful things)
2. Calling LLMs 'AI' is part of the grifting hype that has been wildly prevalent in the US corporate space. It's very natural and also very good that there is now backlash to and scepticism around all of the hype that is has been generated over the last 5 or so years
Finding a counterexample that humans have failed to find for 85 years is a marketing stunt?
[flagged]
If we don't ask for proof when someone claims something, the next few years are going to be rough for everyone. We can't trust anything nowadays
They have provided a literal mathematical proof.
5 replies →
Why?
There will be more and more mathematical proofs provided by LLMs as they improve. If you assume every one is just a marketing tactic, you'll drive yourself insane.
1 reply →