Comment by pveierland

The following is a concise and good explanation of necessary knowledge of information theory:

http://colah.github.io/posts/2015-09-Visual-Information/