Comment by inigyou

But... These are also literally glorified autocomplete. Autoregressive language models are all autocomplete. It turns out that advanced enough autocomplete can do interesting stuff.