Comment by VHRanger

There's already methods that attempt that.

It works for images because diffusion models leave artifacts, but doesn't work so well for text.

Text is an incredibly information dense data format. The diffusion artifacts kind of sneaks into the "extra data" in an image.

The other part is that GPT style models are effectively explicitly trained to minimize that entropy you're mentioning.