Comment by AnthonyMouse
14 hours ago
> Doesn't it make sense that there are some technical questions that are dangerous to supply an answer to?
This has a simple answer: No.
Here's Wikipedia:
https://en.wikipedia.org/wiki/Nuclear_weapon_design
Everything you need to do it is in the public domain. The things preventing it have nothing to do with the information not being available. The main ones are that most people don't want to be mass murderers and actually doing it would be the fast ticket to Epic Retaliation.
Meanwhile the public understanding how things work is important to the public debate over what to do about them. How are you supposed to vote on public policy if the technical details are being censored? How can anyone tell you that a ban on electric car batteries isn't advancing the non-proliferation of nuclear weapons if nobody is allowed to know how they actually work?
Suppose you're an anti-racist preparing for a debate with a racist. You want the AI to give you all the strongest arguments the racist could use so you can prepare your counterarguments in advance of the debate. Should it refuse? Of course not, you're doing nothing wrong.
Why do we need to build totalitarian censorship into our technology? We don't.
> The main ones are that most people don't want to be mass murderers and actually doing it would be the fast ticket to Epic Retaliation.
The main thing preventing random nutcases from making nuclear weapons is they don't have access to the required materials. Restricting the instructions is unnecessary.
It would be a very different story if someone discovered a new type of WMD that anyone could make in a few days from commonly available materials, if only they knew the secret recipe.
> It would be a very different story if someone discovered a new type of WMD that anyone could make in a few days from commonly available materials, if only they knew the secret recipe.
It would need even more to be public. Suppose it was easy to make a biological weapon. You wouldn't be able to effectively censor it anyway and trying to would leave you sitting on an apocalypse bomb waiting for it to leak to someone nefarious or get independently rediscovered before anyone else is allowed to discuss it. What you need is for knowledge of how it works to be public so that everyone can join in the effort to quickly devise countermeasures before some nutcase destroys the world.
Moreover, if something is already public enough to be in the AI training data then it's already public.
Your plan is to release the secret recipe that anyone can use to make a WMD in a few days to absolutely everyone and hope someone comes up with a countermeasure before some nutcase or terrorist decides to try out the new WMD?
The odds of us inventing and deploying countermeasures to a new bomb or chemical weapon or biological agent in a few days is miniscule. You're gambling with terrible odds to uphold a principle in a hypothetical scenario where it's totally impractical. What happened to responsible disclosure, where you fix the vulnerability before disclosing it to the public?
5 replies →
Not quite a nuke (just try obtaining enough uranium ore) but there are some fairly dangerous things a determined nutcase can make without drawing suspicion.
Example determined ned nutcases include Aum Shinrikyo, who tried anthrax, botox, and nukes before succeeding with sarin gas (thank IG Farben!) among other things.
It's a fascinating (if troubling) story: https://en.wikipedia.org/wiki/Tokyo_subway_sarin_attack#Back...