Comment by joe_the_user
2 years ago
Even if the models don't collapse, it seems intuitively obvious that you can't get more "knowledge" out of synthetically generated data than went into it's generation.
2 years ago
Even if the models don't collapse, it seems intuitively obvious that you can't get more "knowledge" out of synthetically generated data than went into it's generation.
A simple counter example: Bootstraping in statistics or any resampling method for that matter, can help us understand things about the underlying distribution, even though we do not know it.
Update: typo
There's other domains where you don't really need much input data to theoretically allow for complex deductions.
The best example is probably maths. A sufficiently intelligent AI could probably create every proof that will ever exist with just a small amount of input data. Similar deductions could be possible in physics, economics, computer science etc.