Comment by jacquesm

Let's do a little experiment. I make a device. We'll call it 'H'. H is a little box. It has my name on it as the creator and a single ethernet port.

We connect H to the nearest network connection and wait for a bit to see what will happen. But because H is just a box there is no outward sign of it doing anything. Meanwhile H is hacking other people's properties left right and center.

If you didn't know that the technology in the box was LLM based would you still consider the box to have gone rogue or would you say that I had created a device with the intent of doing harm?

Because - to me at least - there is no difference. I don't care what is in the box, all I care about is the outcome of deploying a box which you know has the potential to start hacking stuff. It is entirely on me: (1) as the creator of a weapons grade hacking device and (2) as the person deploying such a device (3) as the person who is negligent in ensuring that the device does not harm others (4) as the person continuining the development of such a device after its potential to do harm is known to make sure it does not go out of control. I should be 100% liable for deploying this device with my name on it.

Intent is one angle here, they may not have intended for this to happen, and you'd hope so (though you can't even be 100% sure about that). But the negligence factor here is at least as large, if you know that you are making dangerous stuff the onus is on you to keep it secured. The power and the nature of the models is entirely secondary to the degree to which the purveyors of these models are losing control of their own creations.

Meanwhile, the models that they are warning us against are fairly well behaved and seem to be far less dangerous. But of course for those models that those naughty Chinese distilled from their we need regulation. How they manage to keep all this cognitively dissonant material in their heads is a mystery to me.