Chapter 2: Deep learning

Diffusion: noise forward, denoise backward

Diffusion models are a leading way to make images today. They learn to remove noise one small step at a time. In interviews you may need to write their training loss and one sampling step, and to explain how a text prompt guides the image.

20 minInteractive lessonChapter 2: Deep learningLesson 19 of 20

What you will learn

  • Sample $x_t$ directly from $x_0$ with the closed-form forward process
  • Write the noise-prediction training loss
  • Run one DDPM reverse step and explain classifier-free guidance

Key terms

$\beta_t$
the variance of the fresh kick added at step $t$
$\bar\alpha_t$
the product of the keep shares up to step $t$
$\varepsilon_\theta$
the network's guess of the noise inside $x_t$
DDPM
denoising diffusion probabilistic model, the method in this lesson
guidance scale $w$
how far classifier-free guidance pushes the noise guess toward the prompt

Prerequisites

Take From autoencoders to VAEs: the ELBO and reparameterization first. It comes right before this lesson in the course.

Where this lesson picks up: In the VAE lesson you wrote a sample as mean + scale × standard noise, so you could train straight through a random draw. That left a question: can the same trick help a model that makes images from pure noise?

What comes next

Sampling takes one network call per step, from $T$ down to 1, so is that cost worth it next to a VAE or a GAN?

Next lesson: VAE vs GAN vs diffusion: quality, diversity, cost and FID

More lessons in this part

Take the lesson

Short cards, one idea at a time, with quick checks as you go.