Comment by minimaxir

The point of this post is that this is explicitly not the case. If the metric is measured, the agent finds a way eventually (around 5 total tries typically unless it gets stuck), and learns from iterations where changes caused a regression after a revert.

In one case I used a made-up metric (since I didn't know the exact name or if it existed) and it somehow optimized that too.