Comment by demibabs
5 hours ago
Is this an example of the bitter lesson? Could diffusion models not be this good with equal scaling/compute as LLMs?
5 hours ago
Is this an example of the bitter lesson? Could diffusion models not be this good with equal scaling/compute as LLMs?
Well the LLMs are much bigger than diffusion models so I think that's the bitter lesson. You could scale compute for diffusion models though.
Well no, the bitter lesson isn’t the scaling laws themselves. It’s that approaches which can take advantage of scaling laws will ultimately beat ones that can’t.
You learn something new every day! thanks