Comment by serf

> Knowledge how to manufacture chemical weapons at scale is regulated as well.

Kinda, but also no.

I learned two distinct ways to make a poisonous gas from only normal kitchen supplies while at school, and I have only a GCSE grade B in Chemistry.

Took me another decade to learn that specific chemical could be pressure-liquified in standard 2 litre soda bottles. That combination could wipe out an underground railway station from what fits in a moderately sized rucksack.

It would still be a horrifically bad idea to attempt this DIY, even if you had a legit use for it, given it's a poisonous gas.

I really don't want to be present for a live-action demonstration of someone doing this with a Spot robot, let alone with a more potent chemical agent they got from an LLM whose alignment is "Do Anything Now".

This knowledge isn't regulated.

Anyone with a degree in chemistry can successfully synthesize chemical weapons. This is all public domain knowledge and the chemistry is relatively simple. The technical execution is the hard part but many, many people have these lab/engineering skills. Delivery systems are the hardest part but those are military implementation details and therefore non-public.

It is the same with explosives. Anyone with chemistry skills could synthesize high-performance military explosives, it isn't difficult. Nonetheless, bombings tend to be low-grade explosives like ANFO or garbage explosives like TATP, because the people with the skills aren't the same people that do bombings.

As a chemist you are required to be knowledgeable in these things in part because it is relatively easy to inadvertently synthesize chemicals with rather dangerous properties. Part of the job is knowing what to not do for safety reasons.

Right now LLMs are trained on publicly available information, so if that knowledge is guarded, and if an LLM can provide it, then it's not guarded very well.