Comment by williamcotton

> Compression and GPT are both mathematical processes they aren't the same process;

They're not, but they are very related! GPT has a 1536 dimensional vector space that is conceptually related to a principal component analysis and dimensional reduction in certain compression algorithms.

This does mean that neural networks can overfit and be fully reversible but that is hardly their only useful feature!

They are also very good at translating and synthesizing, depending on the nature of the prompt.

If given an analytic prompt like, "convert this baseball box score into an entertaining paragraph", ChatGPT does a reliable job of acting as a translator because all of the facts about the game are contained in the box score!

But when given a synthetic prompt like, "give me some quotes from the broadcasters", ChatGPT does a reliable job of acting as a synthesizer, because none of those facts about the spoken transcript of the broadcasters is in the prompt. And it is a good synthesizer as well because those quotes sound real!

Terms, etc:

https://williamcotton.com/articles/chatgpt-and-the-analytic-...