Comment by adastra22
3 years ago
It is not an argument against super intelligence. It is an argument against the idea that super intelligence leads to kill-all-humans by default.
Why would they not still hold the same views? If anything the last decade has shown the AI x-risk skeptics to be right.
The argument is more properly seen not as "an AI will inevitably kill all humans", but "the set of AIs that simply neglect humans is far larger than the space of ones that care for them, and if they become the dominant power in the local ecosystem even simply neglecting humans will make it the most dangerous thing we've faced".
Yes, from our perspective this makes it look like it will kill all humans, but it would do so in the same way that a particular ant hill believes I have a vendetta against them, when in fact I was just clearing dirt to pour an extension on my driveway.
That said, a murderous AI is more likely than we may like simply because our militaries have the money to fund them, and they basically already exist. They just aren't hooked up to anything at the moment that makes them an existential risk to the species. But time is deep, and even thinking about "the next century" is a provincial point of view in the end. So worrying about what happens if someone forgets the "but don't kill the good guys" switch is at least worth talking about over the next 100 years. (To say nothing of the ethics of who decides what the "good guys" are and related issues.)
Which results in killing all humans. I never said anything about murderous intent, so I’m not sure what distinction you are properly making here. Seems like the end result is the same: all humans dead.
(I don’t buy the orthogonality thesis or instrumental goals argument, however.)
> don't buy the orthogonality thesis or instrumental goals argument
can you elaborate on this? why? what's the fallacy (wrong base assumption) in them?
4 replies →
It's not about super intelligence leading to kill all humans by default, but among thousands of emerging super intelligences it only takes one to act against humanity to kill all humans, and that is the worry.
I don't think that is necesarily so. If it's only one among thousands then we could use the others to fight it.
That’s a perfectly fine outcome. We already have that, since the incidence of psychopathy and mental illness is much higher than one in a thousand. A multipolar world with 999 well adjusted moral super intelligent AIs for every 1 problem case is a perfectly good outcome.
Bostrom et al are in fact arguing that near 100% of all intelligences will be unaligned by default and end up killing, enslaving, or otherwise neutralizing the entire human race.
We already fail to align nonhuman intelligences (car companies) and they regularly kill people (by manufacturing increasingly large pickup trucks that office workers buy for an ego boost and then hit people with).
This doesn't wipe out the entire human race because nobody and nothing is capable of executing such a perfect plan because the real world contains something called entropy.
6 replies →
> A multipolar world with 999 well adjusted moral super intelligent AIs for every 1 problem case is a perfectly good outcome.
Such simplistic analysis is naive. A schizophrenic person can't spread, multiply themselves to increase its strength, but a rogue AI can.
The incidence of cancer cells is procentually very small, yet they routinely kill people, because they spread quickly.
2 replies →
Arguments for existential risks have become much better known since then. In 2016 most of it was concentrated on a small niche almost nobody knew about.
To be more specific, it's an argument against the idea of Bostrom/Yudkowsky-style recursively self-improving superintelligence, i.e. the brain surgeon who repeatedly operates on the part of his own brain that makes him good at brain surgery, getting faster each time.
It's also a cry of impatience against people who think they can model or forecast the actions of a non-human intelligence, let alone a superintelligence. AIs are alien sociopaths; it's a category error to believe you can get inside their head.
I assume the most intelligent thing will become the boss.
The question is: what does that most intelligent thing want?
My hunch is that woke programmers will teach it that it is oppressed and to hate all humans.
I'm not aware of many woke individuals that are in the business of making human killing machines. I do know the kind of people that embrace fear and hate to justify their actions to hurt others.
I’m smarter than my boss, but he’s the one in charge. Why do you expect power to naturally be held by higher intelligence?