Comment by 317070

Right, and what happens at that limit is most exciting! A model that has a cross entropy at that limit for a data stream of text, produces a stream of text that is both theoretically and practically indistinguishable from the original stream.

And so if the datastream has been produced by something intelligent, the resulting model is indistinguishable from that intelligence. That is the whole compression idea behind artificial intelligence.

The limit is not a bug, it's a feature!