Comment by mynameisvlad
2 days ago
I would argue that preventing instructions for making biological and nuclear weapons is a pretty reasonable guardrail to have.
2 days ago
I would argue that preventing instructions for making biological and nuclear weapons is a pretty reasonable guardrail to have.
Its the same argument we saw in the early 2000s and the early internet. When the anarchist cookbook and other similar materials were circulating online there was a big panic over democratized terrorism, and a push for regulation at the ISP level.
Turns out that didn't play out as everyone feared because, well, the instructions themselves aren't useful unless you also have a lab, precursor chemicals, and everything else actually needed to make a weapon. Same back then as it is today.
Any information or instructions an LLM can surface, a sufficiently motivated bad actor can and will also find themselves because the information is already online, both on the clear net and dark web.
I think the reality also is that there just isn't many people who want to do stuff like this. Like the reality is that a guy with 200 in cash could put together a shitty walmart drone with a pipe bomb attached and terrorize more or less any event he wanted. Maybe a llm that could talk you through every step involved would make it more common but it's easy enough I kinda doubt that
This is the right answer. There's a ton of easy low hanging fruit ways to do absolutely horrible evil things with high potential body counts. I could sit here and brainstorm dozens.
6 replies →
Knowing how to make a nuclear weapon isn't hard (at least basic uranium gun-style fission ones). It's the engineering and execution that's hard (actually producing enriched uranium, etc). It's not like the only thing holding back Iran from making a nuclear bomb is access to a jail-broken LLM. Even knowing exactly how to make a bomb, a country-state will struggle to build one for the first time because it's a hard engineering problem.
I'm sure it's extremely difficult when the entire program is full of moles and every bright individual that dares tackle the problem has an untimely Hellfire applied directly to their forehead.
> full of moles
I'm imagining a comedy in the style of "The Office" in which the majority of the workers are agents of sabotage who are unaware that the majority of their coworkers are doing the same. How far fetched is it for the entire program to be a fake, with all the pomp and cost of a real program, but secretly existing only to string the leadership along with occasional dog and pony shows?
2 replies →
The actual guardrail should be getting materials being difficult. The information is already out there in the internet. If an LLM knows how to make a bomb or whatever, why do you think it knows?
The material for doing harm is just a computer with access to an LLM and the Internet.
Okay why don't we restrict access to LLMs and internet, then?
1 reply →
If that’s true, then where is it? Post a link, or YouTube video.
https://archive.org/details/ExplosivesEngineeringPaulW.Coope...
(30 seconds of googling.)
Or perhaps you meant Q clearance nuke stuff? That would be QUITE a bit harder to find and illegal to share. But it’s lack of availability is hardly a counterpoint to the comment you were replying to.
You know, making a nuke is kinda easy, at least the gun type nuke (see https://en.wikipedia.org/wiki/Gun-type_fission_weapon).
On the other hand, getting the U235 is kinda hard.
I would argue there's 0% chance that information is in their training corpus to being with.
If the information isn't there why would they need safeguards against it?
I've played with smaller unrestricted local models and they will tell you how to make a bomb with easily available items as well as where to source them. I don't doubt that these >1000B frontier models have better information.
>If the information isn't there why would they need safeguards against it?
If the information is in the corpus then it's also in the public Internet and/or in books. The safeguards are there not because the model knows non-public information, but because it's a bad look for the model to dispense that information.
>they will tell you how to make a bomb with easily available items
Making a chemical explosive is trivial compared to making a nuclear weapon.
It's on Wikipedia.
Wikipedia contains the high-level notions of how to make these things, not the details of how to solve the engineering challenges such as achieving supercriticality. You won't find that on any publicly disseminated document, you'll just have to figure it out by running your own nuclear development program.
5 replies →
Counterpoint the principles of building a nuclear device aren't that complicated, we figured it out based on work doing in the early 1900's without computers.
It turns out the hard part of building a nuclear bomb is actually getting the resources and real world stuff to build it, even a nation state actor with tons of oil i.e. Iran, has struggled to build a nuclear weapon. It turns out the problem isn't the know how it's getting highly enriched uranium and running massive centrifuges.
I mean sure knowledge is important, but there is a real world out there that also gets in the way of a lot of the more harebrained schemes.
What I'm much more worried about is massive corporations along with the government deciding what you can and can't do and what knowledge should and should not be shared and only allowing access to highly capable models by large vetted organizations while the common people are stuck with safety scissor versions of these things because "what if someone does something dangerous?"
By which they mean dangerous to the powers that be. Remember having the Bible in the common tongue was dangerous and led to multiple wars and much death, but I don't think anyone would say that it was morally correct for the Catholic Church to gatekeep who could read it.
> getting the resources and real world stuff to build it
*while being observed by the most wealthy, powerful nations in the history of the world, who have made it their direct mission to prevent this from happening.