Comment by againstapples

Not a doomer but I try not to be a denier, either. These are hugely impressive results. I do not see the technology plateauing at the current level (though I am dubious about an infinite exponential growth).

Before I cope, I’ll note that there are plenty of “doom” scenarios that do not require any improvement in capabilities from what we had before this latest unreleased model. We’re at the point where a determined bad actor with enough compute could compromise critical infrastructure in a way that results in casualties, where this actor would not have been capable of such without LLMs. This may not sound like Skynet, but I don’t see why it makes a difference if I’m one of the casualties.

With that in mind, here is the cope: first, mathematics is an inherently verifiable domain. An LLM can use tools to determine with absolute certainty whether it is correct, and an independent third-party could review and confirm. All of this can be done without any interaction with the physical world or with other minds.

Second, OpenAI is able to marshal compute at a scale that an individual mathematician can only dream of. It’s possible that these problems were lower-hanging fruit (in relative terms), such that they could be resolved simply by throwing a ton of compute at the problem guided by an intelligence that is not itself remarkable in comparison to a human.

Third, none of these problems are solved in a vacuum - the reason OpenAI chose these problems is that they are widely discussed and many people are working on them. It’s possible that someone else was close, and OpenAI only contributed the finishing touches. (This wouldn’t need to be plagiarism, to be clear - people publish their work!)

I would like to know how much of the progress comes from effectively combining existing research programs plus massive persistence, and how much is AlphaZero-esque RLVR completely independent of training data. Since I cannot get anybody to care about this question (even though I think it's vital for guessing what the future trajectory will look like -- are we going to complete existing research programs or start new ones?), I live in ignorance and wait for the day when the answer becomes clear.

In looking at this over the past hour, I haven't seen clear evidence one way or the other. Some of the stuff is highly unexpected (like the multiplication algorithm), but counterexample-y, and about the rest the professional mathematicians online seem to have a consensus that it's not "breaking through fundamental obstacles". I suspect neither of us is competent to judge that.

I expect AI will continue to be useful on the vanguard of fields like mathematics because it has the perfect conditions for it to shine. There are a lot of discussions and leads to start from and the AI can check its own work and iterate. It can fail hundreds or thousands of times in a day and continue to work with the same tenacity.

Superhuman tenacity is not enough on its own to pose an existential threat. If it showed the same capacity for judgment, inventiveness, and decision making in the messy problem space of the physical world, I would be more alarmed. There have been experiments where an AI is given control of managing something like a vending machine and it always ends up a mess. AI has come a long way, but certain problems seem as difficult as ever.

When AI becomes more capable of navigating practical problems without human intervention, I will start to be concerned. Enslaving humanity will involve taking a lot of calculated risks that tenacity alone cannot solve.

Depends on your definition of doom.

If doom is ending up with grey goo / paperclip maximizers, or SkyNet, then I don't think doing mathematics is evidence of that direction. Partly because LLMs are quite apparently dumb in many ways, and for math specifically, they need a formal verifier (Lean) which "gamifies" math.

If you mean bioweapons or cyberwarfare, there's nonzero risk, but not orders of magnitude worse than other global risks. Climate change, nuclear weapons, monoculture food, etc.

I'm far more concerned about overall trends in AI development and usage. It's accelerating wealth inequality, social isolation, attention capture, surveillance states. If we end up in the Matrix except the admins are humans and the simulation is hyperoptimized TikTok, is that AI-driven doom, or is it just an inevitable outcome of modern tech?

Where some see intelligence, others see token prediction. It's a very good question how token prediction could achieve this, but I think there's a simple explanation. No human can hold more than a negligible percent of all knowledge in his mind at once. LLMs have no such limits and so can reliably connect 'obvious' dots that we miss simply for lack of storage capability.

Well isn't that just semantics? Surely connecting dots in a novel and meaningful way is intelligence regardless of how it's achieved. The thing is that humans didn't get to where we are by connecting obvious dots. Go back to before humans had invented language and when bleeding edge tech was literally that - 'poke him with the pointy end.' Train an LLM on that corpus of knowledge. Even given infinite processing power and infinite time - it's not going to discover the secrets of the atom, put a man on the Moon, or do much of anything besides remix what we'd already done at the time.

I expect there's still much LLMs can achieve simply because of this initial problem. But I expect that they will ultimately start to plateau once these dots have been mostly matched and we reach a point where 'creation' again becomes the missing link. Though even there LLMs will play a major role as tools. For instance Einstein had to spend a significant amount of time in 'retrieval' rather than 'creation' research to develop the field equations for general relativity. If he had access to LLMs trained on all knowledge of the day, he could likely have achieved his goal much more quickly.

I think that people are just really bad at math and coding. Knowledge workers have been taking pride in doing stuff which the average person does not 'get', but we only understand a little bit. That leaves a lot of room for people to get better, or for other jobs (farming, sandwich cafés, mystery novels) to have been already peaked by human ability and less useful to bring in an AI.

'Doom' to me means that any career crashes, we are controlled, everything is hacked, society stops functioning. Yet every part of my day today (except for coding) was done entirely by people.

Finally I think it's easy to make a simple model that everyone has a simple balance sheet, and that people are more expensive so they will all get cut. But the same argument could be made for all US jobs being outsourced, and all in-person engineers, lawyers, and doctors to be rubber stamps for overseas work.

Try using AI for your work, whatever you do. You will quickly understand the limitations.

I’m not concerned because I consider my skills as a software developer to not be based upon my ability to write code, but my ability to analyze problems. In my mind, as a developer, AI tools are just like a higher form of abstraction in a way, which will enable mathematicians and software developers alike to do much more in a shorter amount of time than they used to be able to. It fills me with optimism, more than dread.

What would fill me with dread was if I considered my skills to be tied directly to my ability to write code. Then I would find myself in a similar situation as manual “scribes” probably found themselves in at the time when the printing press was invented.

The main concern I have, personally, is the speed with which all this is happening. It seems that the speed itself is likely to lead to some level of chaos, because it is happening faster than people, institutions and constitutions are able to cope, and it will leave the door open for opportunists of many kinds, including rogue players.

Can I ask you back, what your concern is here? It'll get so good so as to desire to hurt us or is it a misalignment event that you think will lead to disaster?

Or is it simply that you feel bad for Mathematicians.

Non doomer mostly. I think progress will plod along in a Moore's law like way as it has for 75 years since Turing. They will get very good at stuff like math and get gradually better towards things like a robot coming to fix your plumbing where they are currently well below human level.

I kind of believe we'll merge in some way and become something like immortal so sorta anti doom. We're all going to die unless AI fixes it.

Did AI beating humans at chess:

a) destroy chess and make it a pointless endeavour,

or

b) make humans much better at chess.

From a few preprints I've checked, this is not reliant on just extrapolating existing theories but actually shows novel/surprising approaches. Very few people globally could come up with something like this, even when given time and ressources.

So in other words, since deep learning is algorithmic research, we are now in the RSI era.

I see continual progress in technology and for the second time in my lifetime I see the possibility it will accelerate (the first was when the internet entered mainstream)

I've never been more excited. What a time to be alive!

I see it as “if this can be represented in tokens it can be trained in and ‘solved’”. I don’t think there will be a plateau, but there might be issues with how effectively we can represent some things in a tokenized form and still be efficient.

Non-doomer perspective is that it'll figure out LK-99 for us. Among other things that would be great to have.

I wrote something [0] that might answer a bit a few weeks ago. Today's slopdrop certainly is challenging my stubbornness, but everything I've seen so far indicates that these (hugely impressive, world historic) capabilities won't extend past verifiable domains. Math yields especially impressive results because it so broad and deep that essentially no person can know of all of its parts; pretraining and deep search capacity is a huge advantage. Gowers has recently written about these capabilities and gestured [1] toward some human capabilities lacking from the current frontier models, though he isn't convinced they won't develop soon. If you assume we don't get a total mathematical superintelligence (which to me seems already sort of AGI-complete) and only amplify the capabilities we have today, it's not obvious to me that we get takeoff from recursive self-improvement, unless you happen to believe that 1) we can clearly specify what AGI or ASI is 2) all of the requisite ideas are out there and need only be combined and/or optimized.

[0] https://dank.systems/posts/2026-09-15-ai-bear.html

[1] https://gowers.wordpress.com/2026/08/12/what-sort-of-maths-a...

I'm not sure how AI solving math problems is related to "doom", perhaps you could expand on that? To me (a "non-doomer"), it seems like an overall positive.

I'm starting a p(ButlerianJihad) club. I'm not good at organizing, anymore. Might have to hand it over to my agent.

For me a big open question is what sort of progress will we see in robotics. I won't even attempt to speculate, but it does seem hard in different ways than proving math theorems.

I just don’t have that strong of an association between progress and doom. Maybe just naive?

AI performance has always been extremely spikey. It's great at some things and terrible at others.

Why do you think the world to date hasn't been taken over by evil genius mathematicians? Can you extrapolate from your understanding of the answer to that question?

I completely get the doomer POV, but we've somehow navigated all the previous "dangerous" technologies we've created - electricity, phones, internet - every one of those had similar arguments and concerns of danger.

The optimists' argument:-

Politics:- in general, I think many of the problems in the world today are due to misinformation and lack of education. What happens when we start routing things through an ASI that brings data and logic to the table? What happens when politicians can no longer lie without being caught out live on air? In the UK, local authorities are being flooded with complaints and requests from people; for example, some are doing AI-assisted investigations into accounting "errors".

Science:- I just don't see how the current rate of progress doesn't end up in crazy technologies like perfectly simulated human cells, organs and bodies to the point where we can run experiments virtually and solve all diseases in the next few years. This is happening. Perfect weather predictions far into the future, likewise with earthquakes, etc. Solar panel research explosion resulting in huge efficiency gains, to the point where people no longer need to plug their EV in - car surfaces will be covered in solar panels, as will our windows and roofs. Connecting new homes to the grid will be optional - the same way landline phones are no longer a thing.

I just find it very difficult not to extrapolate all the above.

We got this dump of mathematical breakthroughs from one small team in one company with access to this technology. What happens when this SOTA model is available (and it will continue getting better and cheaper) to everyone working on hard problems - every university on the planet starts cranking out AI-assisted research breakthroughs.

They can replace anyone but they can't replace everyone.

The technology can keep going for a long time in verifiable areas. For non-verifiable areas it's going to have a hard time progressing past where a committee of the best human experts in a field would land. For stylistic areas, whatever the AI doesn't do will have cachet because it will look expensive, sort of like how the kids these days view the ugly old school metal braces as a status symbol because you have to pay for them out of pocket (even as by past standards it'd be truly exceptional).

I'm a transhumanist, I want to build god and kill death. To me this looks like we are moving in the right direction.

So basically I think that the future is getting pretty weird because we are building really powerful tools, though these tools are precisely what allows us to prosper in that future.

I think we crossed plenty of lines were we will not get back to.

Software development for example as a task is done. And AI is continuesly reducing the price of more and more tasks every day.

This math breakthrough also shifts something significant: Its now a lot clearer that investment means money into energy to run AI.

Money + Energy = progress

I don't see it plateuing at all. We know how to progress. We broke through a wall we hit. Like the system wasn't able to optimize/automate everything because the tools were not there. It was still cheaper and easier to hire people for a LOT of things.

Now AI fills this gap.

You will see the commodification of everything in the next 15 years. High complex tasks? commodity. Physical labor? commodity.

For me it's a mixed bag.

There will be a lot of job loss unquestionably, in the same way that automation reduced manufacturing jobs and farm payrolls.

At the same time we have to put what AI can do in perspective.

Intelligence is a broad grouping that includes concepts such as knowledge, skill, experience, and wisdom.

AI has incredible knowledge and in many areas approximates experience and wisdom.

But wisdom is harder to formalize than knowledge and skill.

For example certifying a college education relies mostly on the ease with which we can verify/test knowledge.

To some extent advanced degrees try to certify maybe wisdom and experience.

In my very personal opinion, wisdom and life experience should give humans an edge for a while to come.

Additionally, I do feel that the more an individual lacks better than average wisdom and experience, the harder it will be for that person to compete with AI.

Also, on the bright side, the average human will continue to prefer to interact with a fellow human in many spheres. That will also act as an upper bound on AI and robots taking every job.

Either way, I do think this transition will be painful. I don't feel it has to be apocalyptic.

But the world has been an especially volatile place over the last 10 years.

So when you add that existing volatility, to the upheaval from the AI transition, it would not surprise me if the transition results in violence.

But, without the pre-existing volatility, and if humans were capable of generosity and love at scale, I see no reason AI can not be absorbed into society with net gain.

I guess to summarize, I feel this tech should be a net gain and to the extent that it isn't, it will be because of flaws deep inside of humanity itself, not because it had to end in chaos.

In other words, I feel fear, greed, anxiety, and competition -- all our base instincts coming from all sides, will be what determine the end result of AI moreso than AI taking everyone's job.

AI hater here:

I have a hard time taking statements from OpenAI about their own product, that they are trying to sell to people and make money, seriously. I take these statements as they are greatly exaggerated or even straight up lies and propaganda.

That said, I think results like these are mostly annoying more then anything. They spent a lot of money, used up gigartiuan amount of compute, to ruin a puzzle that mathematicians were tackling. I am mostly unsurprised that if you spend a trillion times the energy that a team of mathematicians would, that you get maybe 1.5 times the results. I see a future where that 1.5 times the results may go to 3x but not much more. And if that, then I will be more annoyed.

You need to clarify whether you are a doomer or a denier/truther? Doomer = p(doom). Denier/truther = Ed Zitron.

I think physics will be the limiting factor. Even if something recursively self improves, it will hit a physical wall allowed by circuits, batteries etc. A lot of the fear is that there’s an upper bound we don’t know about, whether it be time or energy, that allows a fast takeoff to occur fast enough that we dong have time to see it coming. I don’t know about that… so I’m not worried at this point.

[dead]