Comment by Pfhortune

They are tools. You hold the human using it accountable. If that means it's the executive who signed the PO, so be it.

Until LLM's I'd never in my life heard someone suggest we lock up the compiler when it goofs up and kills someone, but now because the compiler speaks English we suddenly want to let people use it as a get out of jail free card when they use it to harm others.

You're free to hold an LLM accountable in the exact same way: fire it if you don't like its work.