Comment by IanCal

Because it’s way easier to completely fuck up a system with running arbitrary commands on it while in use than it is by changing your code. It’s a massive step up in power and a massive drop in how much you can scrutinise a change (to zero).

Maybe the llm can carefully craft an exploit that happens when nginx reads some HTML. Maybe it found a way of hiding file system access in an import I didn’t notice.

I can completely destroy a prod service by accidentally not escaping a space in an rm command.

I’m genuinely confused by this question unless you’ve never worked on production systems in a team before. In which case that’s fine and it’s good to learn but there’s going to be a lot of material out there about deploying and safety.