Comment by girvo

> If you charted performance over time, it'd look like a sawtooth

People have, though, and it doesn't show that. I think it's more people getting hit by the placebo effect, the novelty effect, followed by the models by-definition non-determinism leading people to say things like "the model got worse".