Comment by afavour
19 hours ago
It's exhausting to keep with mainstream AI news because of this. I can never work out if the companies are deluded and truly believe they're about to create a singularity or just claiming they are to reassure investors/convince the public of their inevitability.
It's a fairly mainstream position among the actual AI researchers in the frontier labs.
They disagree on the timelines, the architectures, the exact steps to get there, the severity of risks. Can you get there with modified LLMs by 2030, or would you need to develop novel systems and ride all the way to 2050? Is there a 5% chance of an AI oopsie ending humankind, or a 25% chance? No agreement on that.
But a short line "AGI is possible, powerful and perilous" is something 9 out of 10 of frontier AI researchers at the frontier labs would agree upon.
At which point the question becomes: is it them who are deluded, or is it you?
Sure, when you get rid of the timelines and the methods we'll use to get there, everyone agrees on everything. But at that point it means nothing. Yeah, AGI is possible (say the people who earn a salary based on that being true). Curing all known diseases is possible too. How will we do that? Oh, I don't know. But it's a thing that could possibly happen at some point. Give me some investment cash to do it.
If you claim "AGI is possible" without knowing how we'll actually get there you're just writing science fiction. Which is fine, but I'd really rather we don't bet the economy on it.
I could claim "nuclear weapons are possible" in year 1940 without having a concrete plan on how to get there. Just "we'd need a lot of U235 and we need to set it off", with no roadmap: no "how much uranium to get", "how to actually get it", or "how to get the reaction going". Based entirely on what advanced physics knowledge I could have had back then, without having future knowledge or access to cutting edge classified research.
Would not having a complete foolproof step by step plan to obtaining a nuclear bomb somehow make me wrong then?
The so-called "plan" is simply "fund the R&D, and one of the R&D teams will eventually figure it out, and if not, then, at least some of the resources we poured into it would be reusable elsewhere". Because LLMs are already quite useful - and there's no pathway to getting or utilizing AGI that doesn't involve a lot of compute to throw at the problem.
4 replies →
There are plenty of people that argue that you need nontechnological pixi dust for intelligence.
2 replies →
> But a short line "AGI is possible, powerful and perilous" is something 9 out of 10 of frontier AI researchers at the frontier labs would agree upon.
> At which point the question becomes: is it them who are deluded, or is it you?
Given the current very asymptotic curve of LLM quality by training, and how most of the recent improvements have been better non LLM harnesses and scaffolding. I don't find the argument that transformer based Generative LLMs are likely to ever reach something these labs would agree is AGI (unless they're also selling it as it)
Then, you can apply the same argument to Natural General Intelligence. Humans can do both impressive and scary stuff.
I'll ignore the made up 5 and 25%, and instead suggest that pragmatic and optimistic/predictive world views don't conflict. You can predict the magic word box you feel like you enjoy is special and important, making it obvious to you AGI is coming. While it also doesn't feel like a given to people unimpressed by it's painfully average output. The problem being the optimism that Transformer LLMs will evolve into AGI requires a break through that the current trend of evidence doesn't support.
Will humans invent AGI? I'd bet it's a near certainty. Is general intelligence impressive and powerful? Absolutely, I mean look, Organic general intelligence invented artificial general intelligence in the future... assuming we don't end civilization with nuclear winter first...
Asymptotic? Are we looking at the same curves?
Recent improvements being somehow driven by harnesses and scaffolding rather than training?
With that last bit, I'm confident that you're not in ML, and not even keeping track of the things from what's known to public.
> But a short line "AGI is possible, powerful and perilous"
> At which point the question becomes: is it them who are deluded, or is it you?
No one. It is always "possible". Ask me 20 years ago after watching a sci-fi movie and I'd say the same.
Just like with software projects estimating time doesn't work reliably for R&D.
We'll still get full self-driving electric cars and robots next year too. This applies every year.
> We'll still get full self-driving electric cars and robots next year too.
I've taken a Waymo and it seemed pretty self driving.
1 reply →
> I can never work out if the companies are deluded and truly believe they're about to create a singularity or just claiming they are to reassure investors/convince the public of their inevitability.
You can never figure out if the people selling something are lying about it's capabilities, or if they've actually invented a new form of intelligence that can rival or surpass billions of years of evolution?
I'd like to introduce you to Occam Razor
> if they've actually invented a new form of intelligence that can rival or surpass billions of years of evolution?
Human creations have surpassed billions of years of evolution at several functions. There are no rockets in nature, nor animals flying at the speed of a common airliner. Even cars, or computers or everything in the modern world.
I think this is a bit like the shift from anthropocentric view of intelligence towards a new paradigm. The last time such shift happened heads rolled.
Without a doubt, AGI will be invented much faster with a model to copy from. But similar to rockets, first we'll needed basic gunpowder, then refined fuels, all well before purified kerosene, well before liquified h2 and o2. LLM feel a lot closer to gun powder than even solid rocket fuel. (but because I'm exhausted by the hype, I'm gonna claim that is based on nothing but vibes)
1 reply →
You missed the part where I said "truly believe". I'm not saying "maybe they've made it", I'm asking whether they are knowingly deceiving people or whether they have deluded themselves into believing what they are saying.
ah, apologies, I missed that part.
> I'm asking whether they are knowingly deceiving people or whether they have deluded themselves into believing what they are saying.
I'd bet it's both. Engineers/people making it, are drowning in the hype. Combined with the notion of how hard it is understand something when your salary, or your stock options are based on your lack of understanding. I suspect they care more about building the cool thing, than the nuance they're ignoring to make all the misleading or optimistic claims; whichever side you take depending on how much you actually believe of the inevitability... which look exactly like lies if you're not drinking the koolaid. But expected excitement when your life is all about this "magic"