Comment by jswelker
1 day ago
My argument was that it is reasonable to be concerned that human terrorists might use AI to assist with building bioweapons.
1 day ago
My argument was that it is reasonable to be concerned that human terrorists might use AI to assist with building bioweapons.
That's merely an assertion unsupported by any reliable evidence, not an actual argument. In short you're just making shit up.
Well that's the nature of trying to prevent a thing that hasn't happened yet right? What would be reliable evidence except that terrorists already used AI to help build a bioweapon? I'm sure before 9/11 talking about terrorists using commercial airplanes as makeshift missiles seemed like scaremongering too.
I see zero technical reasons why it could not be done, so being concerned about prevention seems pretty reasonable. I'm not saying AI uses robotic arms to build a bioweapon unassisted or something, just that it dramatically empowers bad actors enough to make them capable of things they previously were not.
The last few years have shown me something about human behavior.
Before the thing happens: "This will never happen, it can't happen, you're making it up, stop being a scaremonger".
10 minute after the thing happens: "Of course, this always happened and it's always been this way and we just have to live with it".
We are quickly adaptable, but that may be risky if we snuggle up with death.
What an incredible illustration of human intelligence failing to generalize out of distribution.
It's quite hard to measure until a wet lab gets behind the filter access to benchmark it.
But if you extrapolate from the ability it has in fields that aren't too strictly filtered, it looks pretty scary.
There are arguments against doing that but at first glance it seems like we just don't really know, and we likely won't: if governments decide they're interested in AI gain of function capabilities they won't be broadcasting that or allowing public benchmarks.
> But if you extrapolate from the ability it has in fields that aren't too strictly filtered, it looks pretty scary.
The closest unfiltered analogy to something as complex as chemistry or biology is most likely the softer fields like philosophy, the humanities and the softer end of the social sciences. Most practitioners and scholars in these fields would agree that AI is not nearly as compelling there as it might be in e.g. math, and that's putting it mildly and charitably.
Even coding shows the divide pretty well: AI writes code that manages to work (i.e. achieve its self-assessed functional goals) but the stuff is so unmaintainable that it ultimately poisons the AI's own context leading to mode collapse. This makes complete sense because maintainability is a soft objective that's especially hard to automatically optimize for in the short term, as part of a RL training run. The math folks themselves, too, now faced with a very real threat to their field from purportedly "hostile misaligned AIs", immediately zeroed in on education and exposition as something that LLMs are terrible at; with their abilities in systemizing and theory-building also being very much in question.