Comment by wahern
6 months ago
> a bunch of capability breakthroughs from (well-curated) AI generations has definitively disproven it.
How much work is "well-curated" doing in that statement?
6 months ago
> a bunch of capability breakthroughs from (well-curated) AI generations has definitively disproven it.
How much work is "well-curated" doing in that statement?
Less than you might think! Some of the frontier-advancing training-on-model-outputs ('synthetic data') work just uses other models & automated-checkers to select suitable prompts and desirable subsets of generations.
I find it (very) vaguely like how a person can improve at a sport or an instrument without an expert guiding them through every step up, just by drilling certain behaviors in an adequately-proper way. Training on synthetic data somehow seems to extract a similar iterative improvement in certain directions, without requiring any more natural data. It's somehow succeeding in using more compute to refine yet more value from the original non-synthetic-training-data's entropy.
"adequately-proper way" is doing an incredible amount of heavy lifting in that sentence.
Yes, but: for humans, even without an expert-over-the-shoulder providing fresh feedback, drilling/practice works – with the right caveats.
And, counter to much intuition & forum folklore, it works for AI models, too – with analogous caveats.