Comment by Plasmoid

It's because LLMs are entropy generators. That's not a bad thing for what people are doing.

But to prevent model collapse you need a way to pump down the entropy. Much like in thermo, it's an expensive and slow process.