Comment by ctoth

Your coding agent is also a very real-world version of the paperclip optimization thought experiment, yes. Have you never seen it reward hacking? Editing tests to pass instead of fixing the code?

It knows what you want, it can even tell you, and it absolutely doesn't give a shit.