Comment by aizk
2 months ago
This is a rare instance where feeding this groundbreaking information into an LLM gives _them_ psychosis. I fed this to claude code and watched it verify the result in 7 different ways to be 100% certain, and it was just flabbergasted. Quite remarkable.
I fed ChatGPT the map with no other context, just “tell me about this function”. It did a bit of work finding the Jacobean etc and eventually worked out the implications of what it was seeing. It then proceeded to check the arithmetic 4 times, and then decided to do a manual verification using an ad hoc symbolic checker in case its SymPy had been tampered with.
Skepticism is the flipside of knowledge.
If an LLM has knowledge encoded inside it (and it's hard to argue it doesn't), then cognitive dissonance can be experienced. And once experienced, must be dealt with, especially in longer-running agentic loops.
A friend was joking the other day about sending some messages under a previously-used Slack identity for an agent (since turned off), then asking the agent about the messages.
The agent maintained it hadn't sent those messages (no memory) and then was forced to reconcile the idea that the messages indeed appeared to come from it.
Its extremely-agitated conclusion was that there had been a security breach and the entire network should be locked down.
Another way to look at it is the LLM is, by definition, what's expected to be probable based on the training data and this, by the same definition, is extremely unlikely data to run across. With high uncertainty comes the need to verify until it can level out as "really surprising" instead of "plausible sounding error".
Can you please share the chat log for this? I would absolutely love to see this.
Not OP, but here's Gemini's reaction: https://aistudio.google.com/app/prompts?state=%7B%22ids%22:%...
It's easy to reproduce, I've fed the example in GPT-5.6 Sol Max and it started multi-checking it in all kinds of ways, with multiple symbolic packages then manual computation, then it did extensive literature search on the subject, looked at tens of math websites, extensive arxiv research. this was soon after it was posted, it didn't find the original twits with the finding
2 replies →
Same result. Public share https://claude.ai/share/19fd1a34-d63b-4a16-8d83-60d5b79e7747
It did the multiple verification sequence before expanding to internet search where it found this thread.
> So the conjecture that survived Keller, Abhyankar, Moh's degree-100 verification, and five-plus published wrong proofs appears to have died via tweet during the World Cup final.
I lol’d. I didn’t know Fable was this sassy
1 reply →
..and (you guessed it) before GTA 6.
Everyone using Claude Fable to verify this proof is so funny. If you read the definition of the Jacobian Conjecture and (I am not exaggerating this) have passed a college Calc 3 class, you can just verify the proof yourself in 30 seconds. The problem was very hard to solve but the counterexample is very easy to verify!
--- edit, adding an explanation:
To summarize it, the conjecture says if you have any multi-variable polynomial function that maps an input to an output in the same dimensional space (take for example: F = (x+2, y+2), which maps 2D space into another 2D space), AND that function has a constant-valued non-zero Jacobian determinant, THEN the conjecture is that the polynomial has an inverse, meaning basically you can find a polynomial that turns the output space back into the input space.
Fable provided the example polynomial (which was very hard to do) and the coordinates which if you plug into it, results in two points being mapped to the same output point. This means that the polynomial can't be inverted, because if you have that output point, how do you know which input point it came from?
You can just plug in the two coordinates it gave into the equation and verify that you get the same output point from both. That's the contradiction of the conjecture and it takes 30 seconds.
---
Something something outsourcing of thinking something.
verification also requires computing the jacobian. an undergraduate-level exercise, but harder than verifying p(pt1) == p(pt2).
Something something pedantry.
I like how first it's amazed and doesn't believe it, accepts it, then realizes you not only stole it from twitter but that it's its own proof lmao.
Which exact model does claude use here?
> the "fable" in that tweet is Claude Fable, i.e., this model
Looks like this was also Fable.
“I'll admit the sequence on my end was genuinely disorienting: I verified the determinant three separate ways looking for the error, verified the evaluations twice, ran out of subtleties to check, and only then searched and discovered that the reason it holds up is that it's apparently my own homework — the "fable" in that tweet is Claude Fable, i.e., this model, working with Alpöge.”
[flagged]
Who knew DOES NOT COMPUTE would be an actual thing?
The unexpected part is it does compute!
Captain Kirk defeated multiple evil computers this way
It reminds me of how AI will estimate that a coding project will take "4 weeks" and then proceed to finish the task itself in 15 minutes.
AI models are changing the world much faster than their own training can keep up with.
Can confirm, Claude is flabbergasted.
Gemini just checks the web first it seems, and already references the news.
Kimi doesn't quite believe it.
turn off search
>kimi is having a blast. i turned search back on and found this post from it’s sources cited after i suggested to check out the reaction. best thing is to go to a model with search off and plop it in the session
Deepseek Pro got stuck after munching it for a minute or so and couldn't quite believe it either.
Like the unicorn emoji, but for math? It occurs when the LLM is presented with incontrovertible evidence against something it "deeply believes" to be true.
So, basically human psychology?
Interestingly, even Qwen 3.6 27B was able to verify the solution, but I didn't get any glazing for discovering it. Instead, it thought that someone named Shestakov had already found a counterexample in 2004.
GLM 5.2 whiffed, it insisted the counterexample wasn't valid.
VibeThinker 3B also recognized that the counterexample was valid. But it kept trying to convince itself that it wasn't, over and over, since it's an "unsolved problem." Eventually it just answered "-2."
I told Gemini Pro I woke up after having dreamt that polynomial and asked if it was related to the Jacobian conjecture. It spent some time thinking and referenced this tweet announcement, saying:
"If you truly dreamt about that specific polynomial, you might be mathematically clairvoyant."
In the rest of the answer, it maintained a cautious skepticism about my claim, saying:
"Here is exactly why the math world is currently scrambling to verify the polynomial you "dreamt" about."
I love how it put "dreamt" in quotes.
Must be that LLMs are just as suspicious of humans as some us are of them.
> Matches! This is bizarre. A Jacobian counterexample has been sitting here in a prompt? Wait... is this map a known "fake" counterexample from the literature? Many mathematicians have tried and failed. This specific map might come from a paper or a forum where it was proposed and then debunked. Or... is it actually correct?
Gemma's having trouble accepting it too. A solution?! At this time of year? At this time of day? In this part of the country? Localized entirely within my own prompt?
Inconceivable!
Yes!
Can I see it?
No.
Current LLMs behave very counterproductively around unsolved problems, especially if they learned that humans consider them difficult. This has many straight up preventing themselves from attempting anything...
Haha, I’ve noticed this as well. It’s like they psyche themselves out about how famous the problem is the same way humans do. I gave one Collatz in disguise, and it was finding all sorts of interesting things (but nothing worth a paper) until it realized the problem was Collatz, at which point it just proceeded to find a bunch of reasons why nothing would work from that point on.
I think vibethinker is heavily overtrained on not attempting to solve open problems.
I had a fun time taking some open problems and disguising them algebraically so that vibethinker 3b would work on them. It managed to prove some interesting things that I didn't know and would be publishable, but for the fact that they already have been. :) (though hard to know if this was because it had been exposed to that knowledge even though it didn't reconize the hidden problem).
Under some maskings it would eventually figure out the problem was equivalent to an open problem then immediately shut down.
It also managed to make some false proofs for various things that duped some other more powerful models.
IMO VibeThinker is the most interesting open model since the OG DeepSeek R1. The conventional wisdom has always been that specialization is not very helpful for LLMs, yet it outperforms models hundreds of times larger in its specialized area. It shows that there is a lot of fruit left to be picked, still out of reach but hanging low enough to be worth going back to the barn to fetch a ladder.
If I were a young Turk in this business, I'd drop everything else and figure out how VT3B is so ridiculously good at math.
More anecdata: When I just tried Qwen 3.6 27B (Q6_K_XL) it (ultimately, after a lot of going back and forth) claimed it was not a counterexample and claimed the Jacobian wasn't constant (which I'm guessing is incorrect). It also mentioned a whole bunch of names it attributed the example to, in its thinking trace.
I only asked once, so it might well be inconsistent. I did try asking GLM 5.2 NVFP4 several times, and it returned consistent wrong answers at both thinking and max-thinking levels.
For Qwen 27B, I have better luck with a Heretic-derived 8-bit quant than I did when I was trying to run the various smaller GGUFs.
Connecting it to a Coding Agent seems much better.
I connected DeepSeek in OpenCode and told it that I dreamed of this counterexample. It called SymPy tools to verify it, said my dream was "surprisingly accurate", and suggested consulting an expert in algebraic sets for independent verification.
Also interestingly, I told it: "DeepSeek told me the Jacobian conjecture is false, and gave me a counterexample. How should I treat this?"
He immediately told me that this DeepSeek was talking nonsense. Someone who can give a real counterexample "would not be a bot from an AI company, but a Fields Medal winner."
> that someone named Shestakov had already found a counterexample in 2004.
Qwen has the sprit of a grad student
It must mean Chestikoff. He dedicated his life to intellectual pursuits due to chronic ill health.
There's a certain (unfair) stereotype of Russian mathematicians that this fits.
I've heard about mathematicians going through kind of the same thing when they get a weird proof that ends up being right from some weird source or themselves.
Which is fair, they get inundated with kooky proofs from amateurs all the time and odds are incredibly good that there's some major fatal flaw that the amateur doesn't see. Or in the case of themselves, there's a certain blindness that makes it a little more difficult to critically evaluate your own leaps. In ether case the way it manifests is by going over it many times and many ways, each time more certain that you missed something until you just kind of break. Only then do you publicly start suggesting that there might be something to this new leap.
> they get inundated with kooky proofs from amateurs
Source?
https://www.ufv.ca/media/faculty/gregschlitt/information/Wha...
The citation needed is for the other part, which claims a mathematician got an amateur proof which was correct.
Nobody is reading unsolicited proofs. They are like spam.
Source: personal friend of a “crank”.
I read some thinking traces someone posted on X, and yeah, near psychosis from refusing to believe this simple of a solution had not been found already
This is also just an annoying Claude personality trait
sol medium can't believe its own input and output tokens either despite computing everything itself; this is what it gave me:
> Taken literally, these two facts would make this map a counterexample to the complex Jacobian conjecture in dimension 3: scaling one output coordinate would normalize the determinant to 1 without restoring injectivity. Since the complex Jacobian conjecture is still treated as an open problem, this strongly indicates that the displayed formula has been mistranscribed or contains a subtle typographical error.
quite interesting indeed!
Just take it as more confirmation that LLMs are unintelligent pattern-matchers.
You must never have faced a situation where you can't believe your eyes. It takes a certain level of - dare I say it - intelligence and maturity to consider that it's more likely you've made a mistake than that you've made a huge breakthrough.
In HN terms - it's never the compiler. Yes, very occasionally it might be the compiler, but you're better off assuming it's a bug in your code.
1 reply →
statistical parrot indeed, just like the one which built the counterexample. maybe.
1 reply →
Hmm, which of these is simple pattern matching?
1) refuting the Jacobian conjecture
2) keep repeating the same disproven statement, because your priors can’t be affected by new evidence
[flagged]
[flagged]
I fed it to Google AI Studio, enabling tool execution and disabling web access. It also quickly verified it with SymPy, then went into psychosis.
5 minutes later: all previous chats are loading fine, but the only "Counterexample to the Jacobian Conjecture" chat is not loading.
Well, I'm not a conventional conspiracy theorist. But everyone knows that in every major LLM provider there are hell of hidden guarding systems that mark users and dialogues based on content (for topics about national security, biology, security, adult topics, etc.) - so there is a small chance a CEO of Google is now receiving a dozens of notifications about "ground-breaking results that could be attributed to Gemini, if act quick". So if any of thousands researchers have ever submitted this polynomial to Claude previously, any Anthropic employee can accidentally or intentionally "rediscover" the result of other researcher (and even hide the traces by deleting a dialogue of other user).
> all previous chats are loading fine, but the only "Counterexample to the Jacobian Conjecture" chat is not loading.
This happened multiple times to me with Gemini. For the most trivial of requests, like translating a video into English.
> "ground-breaking results that could be attributed to Gemini, if act quick"
This would be such a dumb thing to do, and so easy to get caught with...
That is not dumb - erasing copyrights is just their business, and even when caught has zero consequences (for them). Providers can use any users input for improving their models, either by consent, or by flagging any dialogue for safety review (nonconsensually), or by training on whatever content they want anyways (obtained via torrents from pirate sites with U.S. court approval).
On top of that, I retried the same question + one simple question, and again, same behavior - second JC chat is loading forever. That's more just a funny observation over Gemini - today this is very likely some internal issue, tomorrow it can be used for plausible deniability against copyright accusations.
2 replies →
> This would be such a dumb thing to do, and so easy to get caught with...
Based on what I see of American politics, it seems like the thing to do, then!
... for the management of Google that is so eager to present AI results that Prabhakar Raghavan proudly reported to a press crowd telling his daughter about James Webb photographing a planet outside the solar system. This is generally accepted as having resulted in a 10% drop in the Google stock price over 2 days.
(the key being this never happened, and won't ever happen: the James Webb telescope doesn't have even 1/1000th of the resolution necessary to resolve an extraterrestrial planet)
https://www.reuters.com/technology/google-ai-chatbot-bard-of...
https://www.wired.com/story/google-openai-gemini-chatgpt-art...
https://www.popsci.com/technology/google-ai-in-paris/
I wish I could say this only happened once.
2 replies →
Well, Gemini is just a bad model in general, you have two variables going on.
>flabbergasted
phatic mimicry.