Comment by jkhdigital

This is really just a way to show how good GPT-2 is at predicting text. If you know anything about information theory, you'll know that the entropy of the information source places a hard limit on how much it can be compressed. If GPT-2 is really good at predicting English text, then the entropy of its output should be very very close to the entropy of natural English text. Thus, using GPT-2 predictions as an adaptive source encoder will achieve compression ratios that approach the information content (entropy) of English text.