Comment by WhitneyLand
2 months ago
If all checks out this is a huge milestone. AI has now solved one of the most famous open problems in graph theory, using an off the shelf model, in one hour.
It might be a better mathematician than most humans at this point. Kind of like when chess software started beating everyone except grandmasters.
What’s left? Proposing and building out entirely new theories and frameworks? Then better than any human? Then alien math results we struggle to comprehend?
>> What’s left?
For example, there's all the problems that the same off-the-shelf model hasn't solved despite OpenAI running it for many hours on them. Don't forget you're only seeing the results of successful runs.
We can estimate that those unsolved problems must number in the dozens, or even hundreds, given the amount of time that passed since the last announcement of a solution to an interesting problem by an OpenAI model: i.e. the unit distance problem which was announced solved in 20 May this year. That's a couple of months, yes? We can be fairly certain that OpenAI have been trying to solve other problems all this time, first because they are hell bent on demonstrating that their models can do maths and second because we just got another result, but it took that long. They were obviously not twiddling their thumbs all this time.
So if OpenAI are running their model on a single proble for eight hours at a time (according to the prompt they released) they could be easily have run a few hundred instances of their model on the same number of open problems 156 times for each instance (53 days since 20 May, with a model running in three eight-hour sessions per 24 hour day). I mean the only restriction is the cost they're willing to pay for the inference.
So yeah, there's a lot left to do still, don't worry.
The 2 most notable/interesting solutions have come from Open AI directly, but most of the 'LLM solves open problem' category didn't and has come from 3rd parties doing their own thing with publicly available models. I don't see why one would assume they're running models on hundreds of problems. Most likely they have a few problems they especially care about that they run on.
What, Erdős problems? It's hard to see how anyone except mathematicians, and then again only a few communities of mathematicians, especially care about those.
Remember back in the day when Deep Mind made AlphaGo? That made huge waves for two reasons: one, it was very well understood by AI researchers that beating expert humans at Go was very hard; and, two, that Go is a game of great cultural significance to literally billions of people outside of academia, albeit mainly in SE Asia.
Now, Erdős? I'm a computer scientist and I had to look up the planar unit distance problem when I heard about it because I had no idea what that was. Because it never comes up in the literature I read. I'm not saying it's not interesting, but it does seem a bit... random? That they started with Erdős problems? I'd have gone for a Millennium Prize problem, first. P vs NP, Riemann, Navier Stokes, those are heavy-weight results that would establish AI as the de facto approach to mathematics for the foreseeable future. Even I would find it hard to raise an objection (imagine that). Erdős can take a number, compared to all that.
I'm not saying they're choosing problems at random, mind. But it does seem like we're only seeing the tip of the iceberg, with respect to what the AI companies are doing internally. That shouldn't be a surprise. That's exactly how research works in general, both in academia and in industry. There is a clear survivorship bias and we only ever get to see the positive results, never the negative ones.
>> The 2 most notable/interesting solutions have come from Open AI directly, but most of the 'LLM solves open problem' category didn't and has come from 3rd parties doing their own thing with publicly available models. I don't see why one would assume they're running models on hundreds of problems.
Actually, that's a good point but it's in support of my contention. If there are random people in the community running LLMs on their own, favourite, maths problems, we should be seeing many more of those solved and much more often, provided LLMs were really as good at maths as OpenAI et al want them to be. There must be literally thousands of mathematicians trying to use LLMs to solve this or that problem that is famous in their community. Where are all those spectacular results?
Or, to abuse Fermi's question, where is everybody?
5 replies →
It's hard for me not to think what's the point. I am a very average, even below average person in times of intelligence. What is even my value or reason to be if I know anything I can do, LLMs can do better? What is even my value both on job market and as a human?
There are smarter and better humans at just about everything you or I could want to do, that's just life. Most of life isn't about comparative advantages, it's about enjoying life with people we like.
I can't enjoy life with my people if I can't eat, drink, and have a roof over my head.
Sorry to be nihilist, but you never had any objective value if you're thinking in these terms.
As far as we know, the universe "just is". There is no universal objective value of human beings, at all, any one of us.
You have to make or find your own value in the universe. I try not to think too hard about the nihilist side and try to appreciate that for some unfathomable reason, I seem to have what I call consciousness - the ability to observe the present and have it superimposed on the past, and what may be the future, leading me to "experience" things. I don't understand it, no-one does (some people suffering from the Dunning-Kruger effect think they do, but they don't), and yet, here we are.
So it doesn't matter to me if machines perform better than I do, because already lots of other people do. Just try to find your own joy or meaning, somehow.
> As far as we know, the universe "just is".
I don't know this. In fact, billions of people around the world don't know this. In fact, all evidence points to the contrary.
You have objective value being made in the image of a personal God. Denying that leads to a lot of pain, namely nihilistic suffering because it's on you to "pull yourself up by the bootstraps" in any endeavor involving your own self-worth.
10 replies →
Do you have friends or people in your life that are also not geniuses? Do you think about them this way? Why or why not?
This is a great question. Thanks for asking it. It really got people talking.
My $0.02: Your value as a person never had anything to do with your value to the economy. It's time to relearn that fact.
Check out some of Tom Hodgkins books for more. Or maybe anxietyculture.com.
Yeah but what is a persons value economically anymore? What skillsets will enable us to continue to make a decent living till we are old?
1 reply →
But it is. My worth is based on what my value is, what I can do. What when I can no longer have any value on the job market?
1 reply →
Sorry, but most of us need to work to eat. This idea that our "value" is has nothing to do with the economy is an idea rooted in deep privilege, in the ability to say -- if I don't like my job, if I'm not employable, I can just retire, and the only problem will be figuring out how to live life afterwards.
AI will never be better than me at appreciating a good sunset.
Claude performs 50 microappreciations per second
Perhaps it will. And perhaps countless animals are already better than us at appreciating a good sunset, yet we do not seem to value them much.
1 reply →
You are a human being, one of the most wonderful thing the nature has ever created, besides all the other living beings and the wonderful earth we live in. Do not tell yourself you have a value just because some company may want to hire you or not.
Companies and industries already use tools and machinery for tasks were once done by human beings. AI is just another tool they will use and it will probably replace human beings from some intelligence related tasks.
However that may bring more disruption to the society if the government in your country do not protect and help people and leave free rein to capitalistic greed.
I'm my opinion that already happened in the US, not by using AI, but merely by using H1B visa to get intelligence worker from abroad. What happened is that the companies are doing great and getting the best smart people in the world but American people and society have been disrupted.
Live your life fully, be good to yourself and to others. Don't worry about the market.
Your value is intrinsic as a human being. We’re capable of love and shared experiences that a machine will never know.
You have inherent value by virtue of being human. Unfortunately it seems like people have forgotten humanism.
Remember that the most valuable human in the world is Elon Musk. What it means to be valuable is to be like Elon Musk. Calibrate your goals accordingly.
Manual labor
Military will always like warm bodies ;)
Sometimes it's easy for me to imagine a bad future like this; where most of the men are forced into military as the only org that still has use for wetware, and women are valued only for their ability to birth new people...
Anti-ai sentiment won't be automated out haha
Even that is being automated now, it was one of the thing being automated very early on in factories..
1 reply →
Well, you provide training data to the savior and our Lord AI. /s
You don't compete with a chainsaw at cutting trees. You decide what the tree is for. Then sell the rainforest for shareholder value. https://youtu.be/UrgpZ0fUixs
And now AIs can do all that
> What's left?
I think humans will be left to propose new conjectures while machines fill out the proofs. I don't know if there are enough interesting conjectures to go round to build new careers, though.
To be able to propose interesting conjectures you need to walk the walk of trying to prove things yourself. That's not great for future generations.
It is the same with anything technical; you need taste and wisdom, which are borne of experience. I think society will want to subsidize this learning lest it deskills en masse.
2 replies →
Isn't it same argument for every tool we've created?
Surely the machines will have superior conjectures soon.
I think so. Mathematics is the least of my worries. I worry about what the machines will want to do.
You say those things like they're a short step away, but that might not be how it works out.
For example, AI has made zero progress in the last few years in surpassing professionals at art or writing. Its prompt-following skill is much better, and sure, it can render hands and text now, but its artistic sensibility is completely stagnant.
I think, and I may be totally off base, that the labs are specifically avoiding art and (non-technical) writing as an endpoint. It's bad PR for them- it calls attention to the copyright question and threatens the 'human flourishing' kind of jobs- and there's no money in it because people prefer art to be human made and there's hardly any money in that anyway.
The difference is that artistic sensibility is largely subjective. This means that:
1. It's hard to measure (and people can disagree about it)
2. It can't really be improved using RL without a human in the loop (which is how math is being trained)
At a certain level, yes, but AI is still so bad at writing that its failures are objective and easily measurable
1 reply →
Art is so entwined with the human condition that generative AI which conjures output from a statistical pass over background noise is bound to be stagnant.
Obviously it can impersonate art, but where creativity and the human story matter artists need not worry.
AI is no match for rapidly shifting goalposts ;)
AI-written stories have won several awards, e.g. https://lithub.com/a-prize-winning-story-published-in-granta...
What are you even talking about? The last few years, AI has made an insanely big jump in capabilities, performance, accuracy. It destroyed the carrier of a ton of writers and can generate images that are good enough to bamboozle people into thinking it's human made, that sounds like an insanely big leap to me, and yes it can be very creative and in music as well, I would bet it beats already 90% of musicians (most musicians are not that competent).
[dead]