Comment by puttycat
2 days ago
> compression tends to increase the entropy of a bit stream.
Does it? Encryption increases entropy, but not sure about compression.
2 days ago
> compression tends to increase the entropy of a bit stream.
Does it? Encryption increases entropy, but not sure about compression.
Yes: the reason why some data can be compressed is because many of its bits are predictable, meaning that it has low entropy per bit.
I can see what was meant with that statement. I do think compression increases Shannon entropy by virtue of it removing repeating patterns of data - Shannon entropy per byte of compressed data increases since it’s now more “random” - all the non-random patterns have been compressed out.
Total information entropy - no. The amount of information conveyed remains the same.
Technically with lossy compression, the amount of information conveyed will likely change. It could even increase the amount of information of the decompressed image, for instance if you compress a cartoon with simple lines and colors, a lossy algorithm might introduce artifacts that appear as noise.