Comment by cubefox
2 months ago
Image diffusion models also use transformers nowadays. Here is the original "diffusion transformer" paper: https://arxiv.org/abs/2212.09748
Earlier image diffusion models used U-nets: https://en.wikipedia.org/wiki/U-Net
Many U-net based models such Stable Diffusion V1.5 modified the base architecture to include self-attention and cross-attention layers interleaved between convolution layers.