Comment by roywiggins
2 years ago
Even without business incentives, the military advantages of AI would inventivize governments to develop it anyway, like they did with nuclear weapons. Nuclear weapons are inherently unsafe, there are some safeguards around them, but they are ultimately dangerous weapons.
If someone really wanted to use nukes, they would have been used by now. What has protected us is not technology (in the aftermath of the USSR it wasn't that difficult to steal a nuke), but rather lack of incentives. A bad actor doesn't have much to gain by detonating a nuke (unless they're deranged and want to see people die for the pleasure of it). OK, you could use it as blackmail, which North Korea essentially tried, but that only got them so far. Whereas a super AI could potentially be used for great personal gain, i.e., to gain extreme wealth and power.
So there's much greater chance of misuse of a "Super AI" than nuclear weapons.
Sure, that just makes the military incentives to develop such a thing even stronger. All I mean is that business incentives don't really come into it, as long as there is competition, someone's going to want to build weapons to gain advantage, whether it's a business or a government.
Dangerous weapons are not inherently unsafe.
Take a Glock, for example. It is a deadly weapon, designed for one thing and one thing alone. It is, however, one of the safest machines ever built.
The very existence of dangerous weapons, be they nukes or handguns (or swords or tanks), makes the world less of a safe place than if they didn't exist. Existence is pretty much the most inherent attribute anything can have, AFAICS, so yes: Dangerous weapons are inherently unsafe. (Take that Glock, for example -- that's exactly the problem, that someone can take it. Might be someone who knows how to remove the safety.)
> makes the world less of a safe place than if they didn't exist
This is false. Handguns and other small arms, for example, cause a reduction in violence as they proliferate through society, due to the fact that the use of force is not the exclusive purview of the physically strong.
There's a reason police in most places carry guns, and it's not to shoot people; it's to keep people from punching the cops in the face. Nuclear weapons have vastly (and I do mean vastly) reduced mass deaths in world wars since their invention.
2 replies →
[dead]
"Safe", in what sense?
I don’t think we should be stopping things from being developed, we just need to acknowledge that externalities exist.