Neural networks

Diffusion model

Imagine learning to restore photos after adding different amounts of static. A diffusion model learns to predict how to remove noise at each noise level. Starting from random noise, a sampler uses that prediction repeatedly to produce a new example.

Training datax₀ = 2Noisy inputxₜ = 1Predict noisetarget ε = −1StartsamplingGaussian noiseDenoisingstepst → t − 1 → … →0New samplegeneratedoutput
Training example: xₜ = 0.8 × 2 + 0.6 × (−1) = 1; generation instead starts from noise and iterates.

How it works

  1. Choose a training example, a random time step and a Gaussian noise sample.
  2. Mix the example and noise according to the noise schedule to form the noisy input.
  3. Train a time-conditioned network to predict the added noise, or an equivalent target such as the clean input or velocity.
  4. At generation time, start with noise and apply successive denoising updates using a compatible sampler.
  5. For conditional generation, supply the condition at each step and optionally combine conditional and unconditional predictions for guidance.

$$x_t=\sqrt{\bar\alpha_t}x_0+\sqrt{1-\bar\alpha_t}\epsilon,\qquad\epsilon\sim\mathcal{N}(0,I)$$

The knobs that matter

noise schedule
sets the signal-to-noise ratio at each training time
sampling steps
trade generation cost against approximation error for a chosen sampler
guidance scale
increases conditional influence but can reduce diversity or introduce artifacts
pixel or latent space
latent diffusion uses a separate encoder and decoder to reduce denoising cost

What interviewers ask

  • How do the forward and reverse diffusion processes differ?
  • What does the denoising network predict during training?
  • How does classifier-free guidance work?
  • How would you build image-to-image generation with a diffusion model?

Common mistakes

  • Subtracting predicted noise once is not a full reverse diffusion sampler.
  • Training and sampling must agree on time steps, noise scaling and prediction target.
  • Increasing guidance or reducing sampling steps does not guarantee better results.

When to use it

Use diffusion for generation or conditional reconstruction when iterative sampling cost fits the application. For strict latency limits, assess the sampler and step budget alongside output quality.

Implement it (12 problems)

Learn it

Interview questions about diffusion model