Comment by rocmcd
3 months ago
It's hard to read Gwern's accompanying article without seeing this for what it is, which is a kind of mania.
I'm sure he means well and is genuine in his aspirations, but what's outlined for GA is framing LLM's as quasi-gods, which they absolutely are not. I wish him the best, and look forward to being proven wrong.
An LLM just solved 10 top tier math problems this week. It seems very likely that in 6 months they will solve 100 top tier math problems, maybe 1 millenium math problem, maybe 10 top tier physics problems, etc.
It's only going to get crazier.
Math is verifiable. LLMs will continue to make strides in such search spaces where all they need to do is try->verify->repeat. You can't train an LLM on how to end the war in Iran or Ukraine, nor how to solve hunger, nor climate change, etc. Real issues.
And LLMs still suck at art and writing.
Three years ago, people were saying "LLMs just generate plausible-sounding text, they don't understand the notion of truth so they can't do verifiable work like math proofs."
11 replies →
Yeah but... Both things are true. LLMs are doing things we would have considered absolute AGI god tier magic 5 years ago. They also still can't count to 100 yet.
1 reply →
If you think all they need to do is "just that" (try, verify, repeat as you say) then I think you're very, very mistaken.
There is no way for the LLM to bruteforce the search space any better than a human. What it can do better, tho, is to make connections between seemingly (for us) unconnected notions and join them, then verify if that's right.
Your view is not only wrong but also condescending in this day and age.
1 reply →
Claude had some pretty good ideas about how to end the war in Iran when I asked it a while ago (The US should sue for peace, I believe was the conclusion).
>An LLM just solved 10 top tier math problems this week
and 30 years ago a computer beat Gary Kasparov, if you'd listened to Hans Moravec you wouldn't be surprised that the first thing that gets automated is intellectual domain expertise.
Things are going to get crazy when it can figure out how to walk into a random house a and brew a cup of coffee, not do math
When we've done both we will have fully automated Erdős.
1 reply →
Suppose LLM solved all math problems. So what? It’s not like diplomacy and war will end. Are humanity’s problems really constrained on our intellectual ability? If anything most evidence points to cultural failing and all AI will do is enable unprecedented oppression due to said skills at intellectual tasks.
As you read this some poor person is starving. Humanity already possesses the ability to identify said person and send them aid. How is AI going to help here?
The ultimate fallacy is that all technological progress will benefit mankind. That will be true until it is not.
All technological progress will benefit an increasingly smaller (and wealthier) slice of mankind.
I have no idea if diplomacy and/or war will end, I just hope we humans get to hang around a few more years
It's an interesting time when the bar has moved to "It’s not like diplomacy and war will end". But that is on the table. Nuclear weapons caused the end of wars as we know them. Great powers no longer directly attack each other. And AI is much more powerful than nuclear weapons, with an equal capacity for damage and a much greater capacity for good.
By the way, I donate monthly to GiveWell and the shrimp welfare project. Do you donate monthly to starving people? Most people don't, and the reason is simple: at the end of the day, most people just don't care that much about helping a starving person far away. They also don't care much about things like that, animals in factory farms, or earthquakes that kill hundreds of thousands. But ideally, we could find a way to empower people such that the minority who do care can make a big difference.
7 replies →
> Suppose LLM solved all math problems. So what? It’s not like diplomacy and war will end.
These are two vastly different statements, and the the war and diplomacy end is almost trivial by comparison, it would absolutely be solved before we solve all math in even the most steelmanned version.
https://dsimanek.vialattea.net/twain.htm
Has an LLM ever intentionally made you laugh? Cry?
Have you?
(I'm trolling, I'm not sure what you're asking or why)
1 reply →
Incoherent question because LLMs don't have intentions.
[dead]
How many maths problems does one have to solve before becoming god?
I think t's not just a random mania; it seems to me more as a vision - that I am not buying, but it doesn't seem entirely unreasonable.
LLM trained on you and fine-tuned on you. It does make sense in a way!
It is not what I want from LLMs, I think... but it might work.
Gwern obviously knows what he talks about. It doesn't seem like a Gas Town-like fever dream.
> is framing LLM's as quasi-gods, which they absolutely are not
Always such certainty in these dismissals. And whenever you dig in, it always seems to be "well my super-human coding assistant made this dumb mistake!", along with a heavy dose of https://mastodon.social/@falseknees/116477394235883790
I don't think anyone who has spent any time around me would describe me as having anything but a pretty clear-eyed view of the current abilities of LLMs, I don't think anyone would seriously consider me to be in a period of mania or psychosis, and it also seems almost inevitable to me that we are a year or two away from LLM-cognition being substantially more competent in almost every regard from human cognition, and therefore in the foothills of the singularity.
The extraordinary claim that requires extraordinary proof is that this will somehow all blow over, not that we're heading towards a singularity here.
Honest question: What _would_ convince you that we are not heading towards a singularity? Like, is this belief falsifiable?
(Secondary question: what do you mean with singularity?)
As an offering of me engaging in good faith, a controversial opinion of mine: I legit think arpanet going online and starting the networking all of humanity into a massive coupled complex system fits the definition of singularity of "the moment after which predicting what will happen becomes hard to impossible", although that of course heavily depends on your definitions of prediction and hard/impossible.
For falsification, I would take a year in which we don’t see an absolute sea change in capability. The last twelve months were not that, in my opinion, nor were the twelve before it. Hell, I’d even throw in “a year in which capabilities obviously improve but at a cost consumers can’t afford”.
“What LLMs can’t do, only humans can” feels like a God of the Gaps situation: people have to keep moving the goal posts because the more recent models keep unlocking more territory that was previously a “well they’ll never be able to do this!” holdout
1 reply →
> I don't think anyone who has spent any time around me would describe me as having anything but a pretty clear-eyed view of the current abilities of LLMs
You are literally arguing that LLMs are quasi-gods in the same comment you claim a 'clear-eyed view' of current LLMs.
That is a statement of faith.
Every bubble has its believers; that's why we have bubbles. There have been so many times somebody very smart has thought we were on the verge of the rapture, the coming utopia, the workers' paradise, power too cheap to meter, flying cars, settling the moon.
Could this bubble finally be the one? Sure. You could finally be the guy who is vindicated. And tomorrow the people I grew up around could be vindicated by Jesus's return to earth.
I can't prove either of you wrong, because you're not making rational arguments. You've had powerful experiences that have convinced you of something. There is no point in trying to discuss it rationally. The only thing people without faith can tell the faithful is what I said 30 years ago when I got out: I guess we'll see.
But my prediction is that we'll see an AI version of the Great Disappointment: https://en.wikipedia.org/wiki/Great_Disappointment
Many will stay faithful. The date of the singularity will keep getting pushed back. There will be another AI winter, but some will always see summer as just around the corner.
I guess we'll see.
[dead]
It really underscores how rationality is just another religion
No, it has some of the nasty trappings of a religion, like keeping women separate until you need to exploit them, but none of the good moral or creative literary parts of religions.
The end goal of AI is to create quasi-gods.
>The end goal of AI is to create quasi-gods.
I think this is what is driving the hype, the mania, and what increasingly appears to be insanely high investment. It's not about the underlying technology, which is objectively impressive, even if we're not certain of the true cost or utility of it. It's about selling the dream of Olympian omnipotence to investors, implicitly promising that IPO stands for Install Planetary Overlord.[1] And who doesn't want to be on the right side of that? (That there is no 'right side' in such circumstances does not seen to occur to anyone involved in the decisionmaking.)
1. Not my coinage, but that of SF author Charlie Stross in The Jennifer Morgue
The term robot came from the Czech language in 1923. The word was coined by Czech author Karel Capek, first used in his play R.U.R. (translated as Rossum's Universal Robots).[7][8][9] The term comes from the Czech word robotník ('forced worker'), from robota 'forced labor, compulsory service, drudgery,' from robotiti 'to work, drudge', from an Old Czech source akin to Old Church Slavonic rabota (работа) 'servitude,' from rabu 'slave'.
Meh. It does all sound rather grandiose, but in this startup economy I'm sure he can at least manage to get acquired.
At least he's pursuing something more novel than yet another "sandboxes for agents".
In this startup economy? Where almost no one is getting acquired anymore and over fifty percent of global venture is locked up in dead weight and liquidity is at an all time low?
Not sure how you can feel sure anyone can get acquired even if their company had a path to profitability, much less for ones that absolutely don't.
Plenty of companies in (or adjacent to) the AI space are still getting acquired and/or finding VC funding, even as funding in other spaces is drying up.