Comment by sajithdilshan
2 months ago
I don't think knowing how to build a bio weapon means one could actually build a bio weapon. It's like saying knowing how to build a nuclear weapon can easily lead to building a nuclear weapon.
Advanced machinery and safety precaution is needed to engineer a virus or a bacteria to be a bio weapon. It's not something you can do in your tool shed at the moment. Also one would need lethal viruses to even start with and that's not something you can order off of amazon. Further, even to build a chemical weapon, the compounds needed are strictly controlled almost in every country and I would assume any suspicious purchase or order would immediately raise a flag or alert in national security service in respective country.
> Advanced machinery and safety precaution is needed to engineer a virus or a bacteria to be a bio weapon. It's not something you can do in your tool shed at the moment. Also one would need lethal viruses to even start with and that's not something you can order off of amazon.
This is laughably misinformed. You can in fact build a bio weapon in a glorified shed if you know what you're doing. However it will be quite involved, requiring experience on the bench and a great deal of attention to small details. In short an LLM can't suddenly morph you into a molecular biology lab tech with 5+ years of experience.
Meanwhile as with any STEM discipline the educational process effectively serves as a screen for being a reasonably well adjusted adult.
> Further, even to build a chemical weapon, the compounds needed are strictly controlled almost in every country
You can synthesize from basic precursors but you will hit the same issue as above. You will need actual experience on the bench and the process of getting that is going to screen out the vast majority of would be bad actors. (Notably it failed to screen out the members of Aum Shinrikyo but that is very much the exception.)
https://en.wikipedia.org/wiki/Aum_Shinrikyo
> This is laughably misinformed. You can in fact build a bio weapon in a glorified shed if you know what you're doing.
This is pure ignorance and or thinking it can be done like shown in TV shows or movies. Any bio-weapon that is effective would be a virus/bacteria that is propagated via air particles like Antrax. That's not something you can build in a shed because without the safety precaution the person creating it would be the first victim of it.
> This is pure ignorance
I have relevant professional experience but do spout off.
> That's not something you can build in a shed
A negative pressure enclosure, filtration, and UVC sterilization can't be built in a shed? On what basis do you make this seemingly absurd claim? Go check out what hobbyist mushroom growers commonly get up to in their back yard.
BSL-3 is far from technically complex. It's just safety critical to an absurd degree thus (rightfully) mired in bureaucracy.
1 reply →
> Advanced machinery and safety precaution is needed to engineer a virus or a bacteria to be a bio weapon. It's not something you can do in your tool shed at the moment.
I think the claim in this report is that it was a state or semi-state actor. They were from "blocked regions" at a "military research institute." So dismissing the threat by solely modeling it as a disgruntled layman rando is not reasonable.
But you totally have to factor in the fact that anything coming out of Anthropic or OpenAI is part of a propaganda campaign to serve their business interests. These threats need to be addressed in ways that cause Anthropic and OpenAI pain (which means damaging their business). Because, FFS, anyone consciously racing to build doomsday devices needs a good, hard slap.
A state or a semi-state doesn’t need an LLM to build a bio weapon if they wanted to. They can directly hire plenty of experts that are morally corrupted enough to do it.
The reason they haven’t done so far is that it would be 100% assured mutual destruction. Us humans share more than 99.99% of our DNA and any virus/bacteria engineered to evade the immune system and kill humans would infect everyone. Such virus is HIV, it wouldn’t care who is the bad or good actors are.
We saw how dangerous even a flu like virus like Covid-19 can be. One could argue that they can make an antivirus for the bio weapon. But the problem is that the bio weapon can easily evolve in few generations and anti virus could be completely useless
Instead we just get good, hard slop
>not something you can order off of amazon.
I mean there are services in which you can order things from wet labs so it's not completely hypothetical.
And red teaming exercises found that ~80% of labs will gladly synthesize known pathogens without safety review.
Source? Would love to read more about this.
2 replies →
I feel like your comment is relying heavily on bad heuristics (the entities capable of using LLMs to create weapons aren't all individuals) and you are making assumptions that the same "strictly controlled" regimes are in place.
It's worth verifying whether the US, OECD countries, UN, etc will have sufficient regulation over suspicious purchases, for example, after DOGE and funding cuts. This will be an ongoing issue as sovereign debts and bond yields squeeze out spending for other government regulation.