Comment by hoosieree
2 years ago
I almost feel like compression and embeddings are duals of each other, but I can't quite articulate it.
Embeddings use fixed-size vectors to minimize the dot product between vectors of similar inputs. Compressors use a variable-length encoding to minimize the overall stored size.
they are in a way. both encode representation of larger amounts of information on a smaller footprint.