Comment by Auracle

I should have been clear - I'm using the Prodigy settings on that page, not the Adafactor one. You set the learning rate to 1 and the scheduler to cosine, but the real learning rate is figured out by the optimizer.