Comment by dorolow

We use large amounts of synthetic data for training at work and have not observed any sort of model collapse when done properly.

Edit: https://arxiv.org/abs/2404.01413 https://arxiv.org/abs/2406.07515