Comment by wvenable
2 years ago
I don't like this analogy; I think why I don't like it is in the intent. With JPEG in the intent is produce an image indistinguishable from the original. Xerox didn't intend to create photocopier that produces incorrect copies. The artifacts are failures of the JPEG algorithm to do what it's supposed to within its constraints.
GPT is not trying to create a reproduction of it's source material and simply failing at the task. Compression and GPT are both mathematical processes they aren't the same process; JPEG is taking the original image and throwing away some of the detail. GPT is processing content to apply weights to a model; if that is reversible to the original content it is considered a failure.
Blurriness gets weird when you're talking about truth.
Depending on the application we can accept a few pixels here or there being slightly different colors.
I queried GPT to try and find a book I could only remember a few details of. The blurriness of GPT's interpretation of facts was to invent a book that didn't exist, complete with a fake ISBN number. I asked GPT all kinds of ways if the book really existed, and it repeatedly insisted that it did.
I think your argument here would be to say that being reversible to a real book isn't the intent, but that's not how it is being marketed nor how GPT would describe itself.
I think that strengthens my point. We consider a blurry image of something to still be a true representation of that thing. We should never consider a GPT representation of a thing to be true.
> Compression and GPT are both mathematical processes they aren't the same process;
They're not, but they are very related! GPT has a 1536 dimensional vector space that is conceptually related to a principal component analysis and dimensional reduction in certain compression algorithms.
This does mean that neural networks can overfit and be fully reversible but that is hardly their only useful feature!
They are also very good at translating and synthesizing, depending on the nature of the prompt.
If given an analytic prompt like, "convert this baseball box score into an entertaining paragraph", ChatGPT does a reliable job of acting as a translator because all of the facts about the game are contained in the box score!
But when given a synthetic prompt like, "give me some quotes from the broadcasters", ChatGPT does a reliable job of acting as a synthesizer, because none of those facts about the spoken transcript of the broadcasters is in the prompt. And it is a good synthesizer as well because those quotes sound real!
Terms, etc:
https://williamcotton.com/articles/chatgpt-and-the-analytic-...
> With JPEG in the intent is produce an image indistinguishable from the original.
Not necessarily, and even if so, if you continuously opened and saved a JPEG image it would turn to a potato quality image eventually, Xerox machines do the same thing. Happens all the time with memes, and old homework assignments. What I fear is this happening to GPT, especially when people just start outright using its content and putting it on sites. Then it becomes part of what GPT is trained on later on, but what it had previously learned was wrong, so it just progressively gets more and more blurred, with people using the new models to produce content, with a feedback loop that just starts to blur truth and facts entirely.
Even if you tie it to search results like Microsoft is doing, eventually the GPT generated content is going to rise to the top of organic results because of SEO mills using GPT for content to goose traffic...then all the top results agree with the already wrong AI generated answer; or state actors begin gaming the system and feeding the model outright lies.
This happens in people too, sure, but in small subsets not in monolithic fashion with hundreds of millions of people relying on the information being right. I have no idea how they can solve this eventual problem, unless they are just supervising what it's learning all the time; but then at the point it can become incredibly biased and limited.
I don't think JPEG wants to produce an image indistinguishable from the original. It wants to reduce space usage without distorting "too" much. Failing to reduce space usage would be considered a "failure" of JPEG, just as much as distorting too much.
JPEG relies of the limitations of human vision to make an image largely indistinguishable from the original. It specifically throws information away that we are less unlikely to notice. So yes, a good JPEG should be indistinguishable (to humans) from the original. Obviously the more you turn up the compression the harder that is.
It's not quite that straight-forward, though, in that there are two competing goals: Small size and looking as similar as possible to the original. We're explicitly willing trade accuracy for size. How much depends on the use, but sometimes we're willing to trade away so much quality that the artefacts are plainly visible. And we're willing to trade more accuracy for size when the artefacts doesn't distract. For some uses compression artefacts are better than misleading about the original, but for other uses, misleading changes would be preferable as long as they don't give fewer noticeable artefacts for a given size.
2 replies →
I think you're overthinking it.
If I ask ChatGPT to explain something to me like I'm 5, it's going to lose some of the quality in its response, in compared to it being written in 1000 words.
But neither response should be a copy of existing text. The intent of JPEG is to produce a compressed copy of an original. The intent of GPT is not to be a compressed copy of the Internet. It's supposed to producing unique results from what it "knows".
This is an important distinction, especially when there are issues of copyright involved.