Comment by orbital-decay

I'm sorry but how exactly is this new? If you want the model to be useful you also need it to be dangerous. You can already tell the model to do anything you want. If it refuses due to alignment training or external safeguards you can try a jailbreak and bypass all that to make it do what you want regardless. All current models are jailbreakable, they lack reliability. Obviously you, a human, will be held responsible for this. The harness designed by you can do the same.

The narrative these guys are trying to push is that the model itself could be smart and non-aligned enough to end up doing something devastating to accomplish something entirely unrelated. It's definitely not what happened here, as it always turns out to be in these cases it's much closer to Clever Hans than to a paperclip maximizer. Still they're trying to squeeze the blood from the stone to support their narrative and justify a monopoly.