Comment by jmward01

I think a better title is 'The world is not static'. I often point this out for gradient descent. We always think of the static world when envisioning gradient descent but the reality is the world is constantly changing and can often actually be adversarial. This means that in the long term gradient descent can actually select for stability and not optimality in a dynamic world (this is where ruts come from I believe). It would be interesting to publish an 'expected halflife' statistic for scientific knowledge, like biological knowledge, that will change over time.