Comment by williamcotton

2 years ago

> Indeed, a useful criterion for gauging a large-language model’s quality might be the willingness of a company to use the text that it generates as training material for a new model.

I don't find this a useful criterion. It is certainly something to worry about in the future as the snake begins to eat its own tail, but before we reach that point, we can certainly come up with actual useful criteria. First, what makes up "useful criteria"? Certainly it can't be "the willingness of a company to use the text that it generates as training material for a new model", because that is a hypothetical situation contingent on the future. So we should probably start with something like, well, is ChatGPT useful for anything in the present? And it turns out it is!

It's both a useful translator and a useful synthesizer.

When given an analytic prompt like, "turn this provided box score into an entertaining outline", it can reliably act as translator, because the facts about the game were in the prompt.

And when given a synthetic prompt like, "give me some quotes from the broadcasters", it can reliable act as a synthesizer, because in fact the transcript of the broadcasters were not in the prompt.