Comment by dvt
12 hours ago
I'm with @nilesh on this one, and not exactly sure how merely the existence of a solution precludes the advancement of human knowledge. If a problem is "solved" (say, symbolically verified) without any insights gained, it doesn't seem very interesting to the profession.
Navier-Stokes is a bit different (because there's a prize attached, so "scooping" matters), but almost all interesting problems don't have any prizes attached.
Humanity is very biased for the culmination of work, considering everything that comes before and after busywork for the lower masses.
Replicating a paper is just as valuable scientifically as publishing it, but how many careers advance through replication?
If we move the goal from "find the solution" to "clear up the LLMs work" that doesn't bode well neither for the attractiveness of the problem nor for the career of the professional that takes the challenge.
> Replicating a paper is just as valuable scientifically as publishing it, but how many careers advance through replication?
A lot. In fields where knowledge is incrementally building on previous work the reason the whole field hasn't collapsed from the replication crisis is that usually the results that are really high impact are replicated in as an initial step in new research building on it. It's almost never the focus of the paper but you'll often find a quick mention in methods/supplemental of some previous work that was verified to be valid by a replication of a key technique etc. you'll have crisis where old tools are found to be problematic and findings end up revisited etc. Plus fields like clinical research where there's an awful lot of focus on replicating findings using staged clinical trials with increasing statistical power to determine if new interventions work - that's driven by regulatory requirements grounded in good science and a lot of people make careers in just that.
In mathematics, finding novel proofs of a given result is often valuable; it may be a shorter proof (demonstrating better/expanded understanding of the problem) or a translation of the problem into a new domain, setting up more cross-domain advances.
>Replicating a paper is just as valuable scientifically as publishing it, but how many careers advance through replication?
I don’t think this is true, especially for novel or unexpected results. I suppose it depends on what you mean by scientifically, and there is a debate in the philosophy of science about what the value of research even is, but a successful replication does not result in substantial updates to one’s beliefs in the way new research does. And if the goal of science is to change our beliefs and bring them closer to what is “real”, successful replications can’t be as valuable as the initial research almost by definition.
From a pure statistical perspective the first scientific paper shouldn’t update your beliefs as much as the independent replication study.
People don’t behave this way, but a high percentage of all papers have known flaws and that goes up even higher when you consider unknown flaws. Replication doesn’t own its own solve the underlying issue, but independent replication removes a huge range of potential issues on top of providing more information.
1 reply →
Hello.
You have created a fraud machine. Why? With no answer checking then why not make up the most fraudulent crap you can get away with?
Examples: A huge portion of recent non-reproducable science papers.
---
Your thinking, along with everybody that's doing this rat race is causing the pumping out of papers with questionable data, but very little to ensure we are actually making correct science.
It is true for mathematics certainly. I would guess it is less true for science per se.
I think successful replications are as valuable as the original research because they're not unsuccessful replications
I find that to be an issue of maturity (focusing only on the climax and not the process). In Japan, where I live, the culture has a greater appreciation for the context & process, not just the moment of victory.
If you examine the consequences of the inversion of the peak, you realise the need for a balanced perspective.
An AI-generated solution always provides two pieces of info:
Maybe the solution is pretty inscrutable, but it's almost always better than nothing.
So, both of these pieces of info would be at least marginally useful for advancing human knowledge.
> An AI-generated solution always provides ... proof that there is a solution
This is only true in the most trivial sense. A solution is a solution, sure... but how do you know it's a solution, and not an incoherent jumble of words? A human has to review and vouch for it.
Just because the AI gives you an arxiv-worthy PDF, or a Lean proof which compiles, doesn't mean it proves what the AI says it does. The AI could give you the same PDF/Lean code and says it proves the opposite, how would anyone know the difference?
You can't advance human understanding unless you produce things that humans can understand.
Not an expert by any means but the assumption here as I understand it is that the arxiv worthy PDF would not be acceptable or meaningful for impossible to understand proofs. And the lean proof would be meaningless unless the specific expression being proven is human understandable as the direct translation of the question the human is asking in formal form. So proving the negation is not a thing but if you make a subtle mistake in translating the statement you want to prove then obviously the QI is going to be proving the wrong thing. And otherwise you're relying on the correctness of lean as a system and on identifying/preventing if the proof is adversarially exploiting bugs in lean to falsely prove things.
> Just because the AI gives you an arxiv-worthy PDF, or a Lean proof which compiles, doesn't mean it proves what the AI says it does. The AI could give you the same PDF/Lean code and says it proves the opposite, how would anyone know the difference?
> You can't advance human understanding unless you produce things that humans can understand.
And you can't advance human understating unless you maintain that understanding.
I can see a version of the junior software engineer problem here: AI wrecks the problems that could train and motivate the next generation mathematicians, so students abandon the field because there's no place for them. The senior mathematicians who can review/vouch/prompt for AI output like Tao retire and die. Then there's no more math that anyone can understand and no more open problems for it to solve.
And that's probably happening already. I've read articles about AI performing the journeyman work that mathematicians cut their teeth on, rendering years of work obsolete, and derailing the careers that work was meant to start.
1 reply →
That's why the solution should be presented in a verifiable formal language, such as Lean. Which is the case with the Navier-Stokes problem.
I might be wrong, but making an assumption that you could learn to read the mathematical output of the AI long before you could write a solution yourself. But hey, what do I know, I'm not a mathemagition.
2 replies →
This is definitely true in an information theory sense: having more knowledge is always better than less knowledge. However, it may not be true in math as a social human endeavor, and having answers without interesting paths to get there may not expand human mathematics in the same way.
If Fermat had a book with larger margins, would Weil have devoted so much time to proving the Taniyama-Shimura conjecture? No one can say.
It demotivates mathematicians. That’s a pretty large negative!
* current mathematicians
Were early in this cycle, we will learn to do more, and exercise our new capabilities more fluently, which in turn will create more skilled practitioners
Consider the abacus, calculator, computer, etc, each of these enhanced mathematicians’ capabilities and thus outputs.
1 reply →
That’s a skill issue.
2 replies →
More of a 'its the journey' rather than the destination type of thing.Since the insights , quirks, tricks and procedures gained along the way allows insights intoother at that moment unknown problem/domains in the future.
As far as researchers sharing their data/notes with the AI hyperscalars looks like that would be coming to an end wihth a mor guild-like structure going forward to prevent their progress being fron-run by the AI labs.
I wonder if it would be possible for researchers and scientists to submit their papers to an organization which would then collect them, submit them for peer review by other experts in the field, and then release them in periodical form ONLY to individuals and organizations who pay a subscription fee in order to read them while suing those who try to redistribute them without permission?
To what end?
Why would society fund mathematicians if they decided to become a guild that hides secrets? They could pursue that as a hobby, but they’d end up like the coders who refuse to use LLMs - rapidly becoming irrelevant and a bit sad from an outsider’s perspective.
Ah but heres the thing , society/gov expects mathematicians to be productive and tries to measure that by awards/publications/citations gained. Within a guild ope or secret they could possibly use a local LLM (even if slow) to accelerate their collective output.While ensuring their credit/publication/citations remain intact rather than with the AI labs taking a lions share of that.
Think along the lines of the Nicolas Bourbaki persona/collective : " was a collective pseudonym chosen in 1934 by a group of young French mathematicians. None of them carried the name alone; all of them carried it together. And under that name, they launched the most ambitious mathematical publishing project of the twentieth century: a series of texts rebuilding modern mathematics from scratch, on entirely axiomatic foundations."[1]
[1] https://abakcus.com/articles/nicolas-bourbaki
Mathematicians will be less likely to work on a problem if there is a solution - even an incomprehensible one.
> Mathematicians will be less likely to work on a problem if there is a solution
Yes, that is Tao's premise, I'm just not sure I buy it. Suppose an oracle existed which could answer any question truthfully. Let's ignore the mechanics of this for now, but it could say things like "the Riemann hypothesis is False" or whatever and we would take it as gospel.
Does this mean that we wouldn't have mathematicians or physicists or computer scientists or biologists anymore? I genuinely don't think so.
I think his point is that AI is not creating new problems. It may solve "the Riemann hypothesis" but may completely fail to posit a "Mythos hypothesis" which is vital to advance the field. In fact, achieving the former may make the latter even harder because it will disincentivize production of human mathematics which has till now been the only source of "interesting" problems.
FWIW this is my understanding of his argument and I am not a mathematician.
8 replies →
But this oracle doesn't just say true / false. It also gives a proof. That makes it much less exciting (not to mention beneficial for your career) to find another one (or even worse, the same one).
4 replies →
> Does this mean that we wouldn't have mathematicians or physicists or computer scientists or biologists anymore?
In the case of mathematicians, I think not as researchers. What would a research mathematician do? I don't think there would be any reason to try to gain insight from proofs that AI made for the sake of understanding. I don't see what that would achieve besides just retaining extremely niche knowledge (which AI or the oracle already does). The whole point of having that knowledge was to build toward novel work which the AI/oracle does. Also, the time spent and difficulty understanding them could be very high but with no payoff besides just understanding them because the AI/oracle would be used to solve all the problems anyway.
Yes, the present developments, and the present approach, mean we will not have mathematicians any more.
1 reply →
I mean, the oracle doesn't really seem so hypothetical right now. And clearly it's going to drastically change these fields, and mathematics, particularly pure mathematics, must change most of all in order to adapt to the existance of a math oracle (or something close to it).
Yes, but presumably they'll work on another problem instead, because they're mathematicians who enjoy doing mathematics.
Is there value lost in them working on problems that don't have solutions instead of problems that do?
Why was there a prize attached to this problem then? What does humanity get out of this being proved?
Honestly, the attitude of the math community is a bit cringe and increasingly I think some of the elite/mystical aura is fading. Rather than a rich fertile jungle where AI can barely chomp through a fraction of the luscious terrain, one gets the sense it's a desert and all the oases are running dry.
This is my question too. If we are all just going “well that sucks” after AI solves this problem, why did anyone care about the problem being solved in the first place?
Is the bummer that we got a solution we didn’t want - that navier-stokes is not always applicable or something, but we hoped it was?
I think the Navier Stokes problem kind of illustrates what he’s highlighting. I think most people even before AI expected that this would resolve in the negative and that you could get finite time blow up. There wasn’t really ever going to be a situation where the resolution to this question, or really any of the other Millenium Prize problems as far as I know, gives some kind of immediate massive practical feedback.
The hope with many of these problems in math is that in trying to prove that, we get some additional insight into why it blew up that could be applied elsewhere to more general PDEs that cannot be easily controlled.
I think the observation from Tao and many others is that when humans solved these problems, the additional insights into intuition and theory building came for free since humans can give expository on what they found hard or what was their own intuition. This is much more difficult or tedious to extract from an AI model. Even when people did have access to the chain of thought, it wasn’t always very helpful to figure out what was the exact thing that made it all click. This is even more difficult how that the CoT are hidden but I would think the sort of difficulty of extracting the key ideas for a human might be worse now with more advanced models.
There’s a long term aspect to this too where we have historically used these problems as markers for the other parts of mathematics but if AI can solve it all, then suddenly this signal is not very meaningful.
Maybe to bring it closer to home. If an oracle just gave you P \neq NP, then this would be generally uninteresting since this was already expected. There’s a deeper question of why that needs to be answered. However, one would hope that creating such a separation would give us tools that allow us to create lower bounds on a lot more problems we do care about and perhaps some bigger insight onto what makes a problem intrinsically hard or easy. These long term considerations are helpful but are definitely more vague. The remarkable part is that AI is separating the part about proving theorems and the “free” insight you get.
The millennium problems is something done by a single institute to motivate progress on known open problems: https://en.wikipedia.org/wiki/Millennium_Prize_Problems
Yes, but why?
AI companies don't share the dead ends and only sometimes a bit of the process toward success so people don't understand what was curious along the way.
Current career structure of mathematicians works partially by looking at whether they have solved novel and interesting problems, or at least done theory-building that can help solve such problems. Many mathematicians are also motivated by being the world's first to solve such problems
Removing this measure suddenly means that academic mathematic norms need to adapt rapidly, and, even more importantly, intrinsic motivation for many mathematicians needs to change rapidly. That is understandably a sea change for the current mathematics community.
> and not exactly sure how merely the existence of a solution precludes the advancement of human knowledge.
You'd be more sure if you read the tweets.
Tao's point is very simple.
1. Working on problems that AI solvers can solve is a waste of human time.
2. We have no idea which problems can be solved by AI solvers...
3. ...Because the AI labs are keeping their negative results secret, and don't tell us which problems they've tried and failed to solve, and why they've failed to solve them (or succeeded at solving others).
There are additional points surrounding it, but that is the thrust of his argument. His issue is not the existence of AI, but the anti-scientific secrecy in how it is used to solve problems. All the incentives around its current use result in closed, uncollaborative work - which while very attractive to a vulture capitalist, is anathema to scientists.
---
He also posits that having a solution to a problem is a small part of the value of solving a problem. What the AI labs are doing is the equivalent of a student turning in their homework, which has 100% of the right answers, but with none of the 'show your work' steps. Those steps are a critical artifact for doing mathematics, because the process of solving a difficult problem teaches us things about other problems.