Comment by somenameforme

Speaking practically your hypothetical is a scenario that requires somebody that is proactively interested in, and theoretically capable of, making a e.g. dangerous virus, yet are unwilling/unable to do so without a chatbot. How many people might this possibly apply to? I think the number is literally zero.

I also don't entirely understand your comment, because your latter parts do not follow from your lead. You're 100% right that somebody who's not extremely capable messing with this stuff is overwhelmingly likely to just hurt themselves. And somebody relying on a chatbot to guide them in dealing with this sort of tech? Yeah, they're gonna win a Darwin Award.

---

I also think there's an entirely different, yet also compelling argument, against censorship. Local LLMs already exist and are advancing rapidly. There will come a time, probably in the relatively near future, when the state of the art big system and a decent uncensored local system will become practically indistinguishable in terms of capability. So not only will people be able to do this locally, but you lose something big in the process.

The reality is that our interactions with LLMs are 100% being actively surveilled, regardless of privacy promises of the companies involved. At the minimum, every chat is making it's way over to the NSA's Utah data center, one way or the other. Some guy trying to do something significantly malicious using an LLM is little more than a gift to the authorities, but this is only true with centralized/online uncensored services. Push people onto local models to do nefarious stuff, and law enforcement is blinding themselves.