Comment by emrah
2 years ago
This makes perfect sense. Compression is about "understanding", that is, representing the input in a way it can be recognized and labeled. When the recognized bits are larger than the labels, voila you get compression. I'm not surprised that gzip could be better at this task than DNN
Wonder if that means other compression algorithms will be able to do even better...
That’s the idea behind Fabrice Bellard’s NNCP, https://bellard.org/nncp/
I think compression is a subset of understanding. When your child starts to speak grammatically correct, they compressed all the language patterns they were exposed to into the grammar rules.
I say subset because understanding is more general. There might be a specific compression algorithm that performs well on floating point numbers. In contrary, the brains and ANNs might be able to compress any input patterns with a worse performance.