Comment by xstas1

Code is not spec. There is an implementation spectrum.

On one end, you have code that can perform only the behaviour explicitly declared in the spec, but has to be thrown away and rewritten for any new or updated spec.

On the other end, you have code that implements or anticipates a wide range of future possible specs including the given one.

The AI can operate on any point on this spectrum, but it's not very good at choosing. The more complex the software, the more such choices need to be made.

When the number of bad choices reaches a certain critical mass, even a skilled engineer becomes powerless to undo all the bad choices, and even a powerful model becomes unable to reduce it back to a coherent spec.