Comment by mycall

6 years ago

If someone could make a billion word dictionary of these words, you could get excellent sentence compression rates out of standard English.

3 comments

mycall

Reply
minimaxir  6 years ago

GPT-2 itself works off of excellent sentence compression (byte-pair encoding for the input and output)

  • mycall  6 years ago

    so for example, a byte-pair input could be "potato salad in rum" and the output might be "potadum"?