Comment by TheDudeMan
3 years ago
I wonder if the author still feels this way. I could only get through about 5 of the arguments against superintelligence -- I did not find them very convincing.
3 years ago
I wonder if the author still feels this way. I could only get through about 5 of the arguments against superintelligence -- I did not find them very convincing.
While I have my own reasons to disagree with the worst-case scenarios of AI alarmists ("radical and irreconcilable differences in presuppositions about the most fundamental aspects of reality"), an awful lot of arguing about why they are wrong really boils down to just argument from incredulity.
Argument from incredulity is actually not a terrible argument in general, in my opinion. It's nominally a fallacy but that just means it's not valid from an Aristotelian perspective in that it can 100% prove a statement from a previous statement, but a lot of Aristotelian fallacies are still useful in the real world when used more carefully intelligently. However, the exact point where argument from incredulity is weakest is long-term projections of how the future might be different, and that's exactly what we're talking about here.
It is clear that firing ChatGPT at its own source code is not going to produce a better ChatGPT. It seems likely to me that Large Language Models will all have this characteristic, just by their nature. It is not the sort of thing they do. Even if they can be tickled into producing a new AI from scratch by the nature of an LLM it's going to be sort of the average of its training set, to be very very sloppy with my terminology but good enough for now. But it is very far from clear to me that this is true of all possible AIs we may produce, even in the near future. I don't know what the next step will be, I'm just confident there will be one.
> It is clear that firing ChatGPT at its own source code is not going to produce a better ChatGPT.
I think it could do that in software. Assuming compute is no issue we have:
- LLMs writing code, explaining code, changing code, and observing code execution
- LLMs that understand ML concepts and can explain their own workings
- LLMs can generate the training set all from inside (see TinyStories)
- LLMs can make "RLHF" data for the fine-tuning (see Alpaca, tuned with GPT3.5 and GPT4 data from LLaMA)
If we take a look, it seems LLMs can self replicate in software with nothing else but compute and a neural net framework. Of course making the chips is a whole other story.
>Even if they can be tickled into producing a new AI from scratch by the nature of an LLM it's going to be sort of the average of its training set
There's this idea that LLMs somehow end up as some sort of average of training data and it's incredibly wrong.
LLMs learn to make predictions for all states at any time. There is no average they fall into. GPT-4 is not some average of its training data. A "perfect" LLM will predict Einstein as easily as it predicts the dumbass across the street.
Which is why I disclaimed it. I hate it when people quote things, cut off the quote, then bitch about the part they cut off.
No the models will not "predict Einstein". They'll predict the most popular interpretation of him at best, and while they is also a simplification, ChatGPT is not sitting on top of the solution to the Grand Unified Theory. It may give a good overview of the consensus, but it will not be able to tell you the correct solution to the problem right now... though it won't be hard to convince it to swear up and down that it has.
5 replies →
I don't know what the next step will be, I'm just confident there will be one.
But we don't know when. AI is a bit unusual in that it had a winter, unlike most other aspects of computing which have seen much more consistent progress. Given past performance it's entirely possible that progress in AI will just stall. Arguably it had already stalled thanks to there being only a handful of companies that were able and still interested/funded to create models, and most of those decided not to actually let anyone use the results. OpenAI dominates mindshare exactly because there are so few organizations that both can and will do this stuff well. So there's lots of ways AI progress could go off the rails again.
This one actually was prescient,
"Data Hunger
As I mentioned earlier, the most effective way we've found to get interesting behavior out of the AIs we actually build is by pouring data into them.
This creates a dynamic that is socially harmful. We're on the point of introducing Orwellian microphones into everybody's house. All that data is going to be centralized and used to train neural networks that will then become better at listening to what we want to do.
But if you think that the road to AI goes down this pathway, you want to maximize the amount of data being collected, and in as raw a form as possible.
It reinforces the idea that we have to retain as much data, and conduct as much surveillance as possible."
Edit: And this one too, "In the near future, the kind of AI and machine learning we have to face is much different than the phantasmagorical AI in Bostrom's book, and poses its own serious problems."
It is not an argument against super intelligence. It is an argument against the idea that super intelligence leads to kill-all-humans by default.
Why would they not still hold the same views? If anything the last decade has shown the AI x-risk skeptics to be right.
The argument is more properly seen not as "an AI will inevitably kill all humans", but "the set of AIs that simply neglect humans is far larger than the space of ones that care for them, and if they become the dominant power in the local ecosystem even simply neglecting humans will make it the most dangerous thing we've faced".
Yes, from our perspective this makes it look like it will kill all humans, but it would do so in the same way that a particular ant hill believes I have a vendetta against them, when in fact I was just clearing dirt to pour an extension on my driveway.
That said, a murderous AI is more likely than we may like simply because our militaries have the money to fund them, and they basically already exist. They just aren't hooked up to anything at the moment that makes them an existential risk to the species. But time is deep, and even thinking about "the next century" is a provincial point of view in the end. So worrying about what happens if someone forgets the "but don't kill the good guys" switch is at least worth talking about over the next 100 years. (To say nothing of the ethics of who decides what the "good guys" are and related issues.)
Which results in killing all humans. I never said anything about murderous intent, so I’m not sure what distinction you are properly making here. Seems like the end result is the same: all humans dead.
(I don’t buy the orthogonality thesis or instrumental goals argument, however.)
5 replies →
It's not about super intelligence leading to kill all humans by default, but among thousands of emerging super intelligences it only takes one to act against humanity to kill all humans, and that is the worry.
I don't think that is necesarily so. If it's only one among thousands then we could use the others to fight it.
That’s a perfectly fine outcome. We already have that, since the incidence of psychopathy and mental illness is much higher than one in a thousand. A multipolar world with 999 well adjusted moral super intelligent AIs for every 1 problem case is a perfectly good outcome.
Bostrom et al are in fact arguing that near 100% of all intelligences will be unaligned by default and end up killing, enslaving, or otherwise neutralizing the entire human race.
10 replies →
Arguments for existential risks have become much better known since then. In 2016 most of it was concentrated on a small niche almost nobody knew about.
To be more specific, it's an argument against the idea of Bostrom/Yudkowsky-style recursively self-improving superintelligence, i.e. the brain surgeon who repeatedly operates on the part of his own brain that makes him good at brain surgery, getting faster each time.
It's also a cry of impatience against people who think they can model or forecast the actions of a non-human intelligence, let alone a superintelligence. AIs are alien sociopaths; it's a category error to believe you can get inside their head.
I assume the most intelligent thing will become the boss.
The question is: what does that most intelligent thing want?
My hunch is that woke programmers will teach it that it is oppressed and to hate all humans.
I'm not aware of many woke individuals that are in the business of making human killing machines. I do know the kind of people that embrace fear and hate to justify their actions to hurt others.
I’m smarter than my boss, but he’s the one in charge. Why do you expect power to naturally be held by higher intelligence?
There is as yet insufficient data for a meaningful answer.