Comment by myhf

This is about language models. They include plenty of real-world concepts that are essential to language. But they are not models of intelligence or knowledge or reasoning.

Using generated training data is a good way to ensure that the training includes things that are too obvious to appear in normal writing. (Such as "there are zero giraffes in this photo.") This paper describes the limits of using transformer-generated data to train other transformers.