Comment by hnlmorg
6 hours ago
You’ve identified that the risks of nuclear weapons are theoretical. ie in theory we could blow up the world even though we haven’t yet done so.
Well the worries about AI are equivalent in that those risks are discussed now because discussing them after they’ve happened is clearly too late.
That’s the thing about risk. There’s no point discussing it after it’s happened and any discussions beforehand can easily be hand waved away as “it’s just a small group of unrelated individuals” or “it’s unlikely to happen to me”.
So yeah, your points are true. But they’re also moot.
The risks of nuclear weapons aren't theoretical. Nuclear weapons have killed people, destroyed infrastructure, and contaminated the environment. The Limited Test Ban Treaty was put in place after radioactive fallout from repeated nuclear weapons tests made people sick.
So in fact we've done exactly what you suggest there's "no point" doing - used the things and then had a discussion after the fact about limiting future use of them.
FWIW, before any nuclear weapons had ever been tested, a risk was identified that the first one might trigger a self-sustaining reaction in atmospheric nitrogen and destroy the entire planet.
Faced with such a scenario, is the prudent next move:
a) blow one up and see what happens, or
b) do whatever you can to be sure it won't happen before conducting the first test, and make sure the confidence in the calculation is very very high
because I vote for b, and so did Teller.
Good point, humanity chose a and the sky in fact did NOT ignite an destroy the planet. The hypothesis was very very far off from what I recall.
But it's good we talked about it.
It's funny that when faced with a weapon that could finish the war, option b was still selected (thanks Teller).
But the people faced with decisions close to IPO, well, we know what is happening.
“Self-sustaining reaction,” like the recursive self improvement?
Or perpetual machines?
Is physics no longer a main subject at schools?
1 reply →