← Back to context

Comment by demibabs

5 hours ago

Is this an example of the bitter lesson? Could diffusion models not be this good with equal scaling/compute as LLMs?

3 comments

demibabs

Reply
moojacob  5 hours ago

Well the LLMs are much bigger than diffusion models so I think that's the bitter lesson. You could scale compute for diffusion models though.

  • demibabs  5 hours ago

    Well no, the bitter lesson isn’t the scaling laws themselves. It’s that approaches which can take advantage of scaling laws will ultimately beat ones that can’t.