Opening Transformers in Diffusion Models for Image Generation and Beyond
Transformers in Diffusion Models for Image Generation and Beyond
- Length
- 1 h 15 min
- Level
- Graduate
- Recorded
- Stanford University2025
Chapter 1 of 11: Before the lecture
Paul follows diffusion backbones from U-Net to DiT, PixArt-α, SANA and MMDiT, comparing them block by block.
Diffusion transformers have replaced the convolutional U-Net as the backbone of many image and video generators. Paul, of Hugging Face, starts from the iterative denoising loop conditioned on text and time step, then shows why giant U-Nets from DDPM, latent diffusion and SDXL scale poorly. He builds DiT from ViT blocks with adaptive layer normalization, then PixArt-α, which adds text encoders and cross-attention. For high resolutions he covers compressed latents, SANA's linear attention and MixFFN, and MMDiT's separate QKV projections for text and image. Later sections treat parameter sharing, structural controls and video. The session belongs to Stanford CS25 and assumes familiarity with self-attention and basic diffusion.
Diffusion transformers have replaced the convolutional U-Net as the backbone of many image and video generators. Paul, of Hugging Face, starts from the iterative denoising loop conditioned on text and time step, then shows why giant U-Nets from DDPM, latent diffusion and SDXL scale poorly. He builds DiT from ViT blocks with adaptive layer normalization, then PixArt-α, which adds text encoders and cross-attention. For high resolutions he covers compressed latents, SANA's linear attention and MixFFN, and MMDiT's separate QKV projections for text and image. Later sections treat parameter sharing, structural controls and video. The session belongs to Stanford CS25 and assumes familiarity with self-attention and basic diffusion.
Denoising Diffusion Probabilistic Models (DDPM)
High-Resolution Image Synthesis with Latent Diffusion Models (LDM)
SDXL: Improving Latent Diffusion Models for High-Resolution Image Synthesis

This lecture
Talk 20 of 21 of Transformers United.
Choose a node to see how it connects to this lecture. What comes before is on the leftat the top. What comes after is on the rightat the bottom. The ring is what it teaches. People and places are above. Other lectures and books are below.
Press a chosen node again to open it.
Your comment is kept while you sign up.Sign in
Simple diffusion: End-to-end diffusion for high resolution images
PixArt-alpha
PixArt-$\alpha$: Fast Training of Diffusion Transformer for Text-to-Image Synthesis
SANA: Efficient High-Resolution Image Synthesis with Linear Diffusion Transformers
Lumina-Image 2.0: A Unified and Efficient Image Generative Framework
EfficientViT: Memory Efficient Vision Transformer with Cascaded Group Attention
SegFormer: Simple and Efficient Design for Semantic Segmentation with Transformers
Stable Diffusion 3 paper
Scaling Rectified Flow Transformers for High-Resolution Image Synthesis
Albert
DiT-Air