Comment by slashdave
2 years ago
Floating point precision is not involved (most LLM models still function after floating-point quantization).
I am puzzled that some find this result at all surprising. You simply cannot generate information from nothing.
I'm not surprised you can't use it to make it better, but one might imagine gradients would go to zero as you fed the model its own output.
No, not even close. Gradients don't come to zero in the first place. Training is never perfect.
Let's restate. I'd imagine you end up in local minima that are difficult to escape using model generated data. So sure, non-zero gradients, but if you plot the gradients, I would expect them to orbit at that point. But it seems like they diverge.
2 replies →