Comment by nylonstrung
15 hours ago
I honestly think the diffusion LLM approach is a dead end
It's telling that frontier labs like Google toyed around with it but didn't invest further even for their most speed and cost sensitive small models
Still unclear for what, if any use cases this is pareto frontier
From what I’ve heard, the issue is more that it’s harder to efficiently share the hardware across diffusion requests, so it’s more expensive to serve.
It sounds like there might be opportunities for local models (not open weight, but actually locally run) to use diffusion for faster responses on weaker hardware that doesn’t need to be shared.
But yea, it’s still a red-ish flag that big labs haven’t invested much in it. I could see Google/Apple getting value of this sort of local model, but maybe there’s enough research behind traditional models that it’s not worth the distraction at this point in time.
You can't think that a small startup versus Anthropic's training setup is anywhere near the same scale to make apples to apples comparisons.
Not sure how the Chinese labs pull it off though using autoregressive models. The secret sauce is probably going to be in the training data.
The main reason Google hasn't switched over to DiffusionGemma is because serving at larger batch sizes loses the speed gains you get from diffusion, and most of the primary use case is serving many users at once off a single device with a large batch size.
If you were to move to on-device low latency... like say in a robot or something, then the story might be different...
Personally, I think it's more that text diffusion is not the ideal driver of an agentic work loop than that text diffusion is a total dead end. I am still hoping to see how it does on authoring and editing with further scaling and optimization. I think the push for AGI has put a bit too much focus on the idea of one general model doing everything.
There were too many unsolved problems with diffusion to commercialize it, where autoregressive loops had a more straightforward roadmap. Doesn't mean the diffusion problems are insurmountable though.
DiffusionGemma was released alongside the other gemma-4 models just a few months ago, so clearly google hasn't abandoned the idea.
K2-Horizon-7B has a diffusion and non-diffusion variant, and they claim the same level of intelligence from both models.