Comment by agajews

you don't need to pick good images manually! it's similar to next-token prediction, many prediction tasks aren't especially interesting, but there are enough hard ones that the model can spend the most time learning from those. the simplest thing scaled up works very well.