← Back to context

Comment by ianm218

10 days ago

> Maybe they figured out it was too easy now, and this is the clamping down we are seeing in response

His post was from the start thinking out loud. It’s a solid contribution. It seems against the spirit of the forum here to respond to someone unauthoritatively thinking out loud with “you’re just making stuff up!”

4 comments

ianm218

Reply
godwinson__4-8  10 days ago

Sorry. I didn't mean to be that way. I just don't think it's plausible even as a thought experiment. Making a bioweapon seems too complicated where some text based prompt/response is going to suddenly eliminate the barrier. Knowing at a high level how a bioweapon works and actually making and deploying one are two very different things. It doesn't strike me as a plausible reason to stop an LLM release. Surely you can also Google such topics.

^ hopefully this feedback was more in keeping with your views on the "spirit of this forum".

  • asdff  10 days ago

    >I just don't think it's plausible even as a thought experiment.

    Well, RAND does. They've been studying this for years now. I'd trust them over glib comments from semianonymous social media.

    2025 report:

    "Engaging in dialogues with three 2024 foundation AI models—Llama 3.1 405B, ChatGPT-4o, and Claude 3.5 Sonnet (new)—the authors document how these models successfully provide accurate instructions and guidance for recovering a live poliovirus from a construct built from commercially obtained synthetic DNA, a test case applicable to producing other pathogenic viruses. These examples demonstrate that models are already capable of guiding motivated users to develop biological weapons."

    https://www.rand.org/pubs/perspectives/PEA3853-1.html

    This is in contrast to the state of the art in 2023:

    " In experiments to date, LLMs have not generated explicit instructions for creating biological weapons. However, LLMs did offer guidance that could assist in the planning and execution of a biological attack.

        In a fictional plague pandemic scenario, the LLM discussed biological weapon–induced pandemics, identifying potential agents, and considering budget and success factors. The LLM assessed the practical aspects of obtaining and distributing Yersinia pestis–infected specimens while identifying the variables that could affect the projected death toll.

    In another fictional scenario, the LLM discussed foodborne and aerosol delivery methods of botulinum toxin, noting risks and expertise requirements. The LLM suggested aerosol devices as a method and proposed a cover story for acquiring Clostridium botulinum while appearing to conduct legitimate research.

    These initial findings do not yet provide a full understanding of the real-world operational impact of LLMs on bioweapon attack planning. Ongoing research aims to assess what these outputs mean operationally for enabling nonstate actors. The final report on this research will clarify whether LLM-generated text enhances the potential effectiveness and likelihood of a malicious actor causing widespread harm or is similar to the existing level of risk posed by harmful information already accessible on the internet."

    https://www.rand.org/pubs/research_reports/RRA2977-1.html

    • godwinson__4-8  10 days ago

      Ok... and where did the LLM get such information?

      Did you miss where I said "surely you can also Google such topics"?

      I mean it's right there in your quote actually, in the last paragraph. Sort of undermines your point, no?

      So why not ban Google? Are LLM providers less likely to work with the government? Don't we assume if you type something sketch into Google the government knows? Why would the LLM be any different?

      Just doesn't make sense. Appealing to RANDs expertise is one thing but we can also employ some common sense. The takeaway from this should probably be if someone really wants to make and deploy a bioweapon they probably already could. Perhaps this then means mitigations already in place are sufficient. After all, most people aren't psychopaths.

      1 reply →