Comment by Cthulhu_

The only way LLM-based software development / production management will be trustable is by actually scaling back what it can and cannot do. Put critical operations in "real" code, so that the LLM can only request a release, triggering a human review of, at the very least, the operation that is about to be done.

Then again, this reminds me of the prompts in operating systems whenever something needs root access, most people just blindly okayed it, especially on Windows since Vista did too many of them even for trivial operations.