Comment by LarsDu88

You can't think that a small startup versus Anthropic's training setup is anywhere near the same scale to make apples to apples comparisons.

Not sure how the Chinese labs pull it off though using autoregressive models. The secret sauce is probably going to be in the training data.

The main reason Google hasn't switched over to DiffusionGemma is because serving at larger batch sizes loses the speed gains you get from diffusion, and most of the primary use case is serving many users at once off a single device with a large batch size.

If you were to move to on-device low latency... like say in a robot or something, then the story might be different...