Chapter 2: Deep learning
Diffusion: noise forward, denoise backward
Diffusion models are a leading way to make images today. They learn to remove noise one small step at a time. In interviews you may need to write their training loss and one sampling step, and to explain how a text prompt guides the image.
What you will learn
- Sample $x_t$ directly from $x_0$ with the closed-form forward process
- Write the noise-prediction training loss
- Run one DDPM reverse step and explain classifier-free guidance
Key terms
- $\beta_t$
- the variance of the fresh kick added at step $t$
- $\bar\alpha_t$
- the product of the keep shares up to step $t$
- $\varepsilon_\theta$
- the network's guess of the noise inside $x_t$
- DDPM
- denoising diffusion probabilistic model, the method in this lesson
- guidance scale $w$
- how far classifier-free guidance pushes the noise guess toward the prompt
Prerequisites
Take From autoencoders to VAEs: the ELBO and reparameterization first. It comes right before this lesson in the course.
Where this lesson picks up: In the VAE lesson you wrote a sample as mean + scale × standard noise, so you could train straight through a random draw. That left a question: can the same trick help a model that makes images from pure noise?
What comes next
Sampling takes one network call per step, from $T$ down to 1, so is that cost worth it next to a VAE or a GAN?
Next lesson: VAE vs GAN vs diffusion: quality, diversity, cost and FID
More lessons in this part
- From autoencoders to VAEs: the ELBO and reparameterization 7 min
- VAE vs GAN vs diffusion: quality, diversity, cost and FID 20 min
Take the lesson
Short cards, one idea at a time, with quick checks as you go.