Neural networks
Generative adversarial network
Imagine a counterfeiter making coins and an inspector learning to spot the fakes. A GAN trains a generator to create samples and a discriminator to distinguish generated samples from real data. Each network changes the learning problem faced by the other.
How it works
- Sample real training examples and random latent vectors.
- Update the discriminator to distinguish real examples from generated examples, holding generator parameters fixed.
- Generate another batch and update the generator through the discriminator, holding discriminator parameters fixed for that update.
- Alternate the updates and evaluate both sample quality and coverage of the data distribution.
$$L_G^{\mathrm{non\text{-}saturating}}=-\mathbb{E}_{z\sim p(z)}[\log D(G(z))]$$
The knobs that matter
- latent dimension
- sets the size of the random input to the generator
- learning rates
- control the speed of each competing network
- discriminator updates per generator update
- changes the balance between the two training processes
- adversarial loss
- changes the signals used to train the discriminator and generator
What interviewers ask
- How do the generator and discriminator train each other?
- What is mode collapse, and how would you detect it?
- Why can GAN training become unstable?
- How would you compare a GAN with a VAE or a diffusion model?
Common mistakes
- A discriminator near chance does not by itself prove that generated samples are good.
- Updating the generator requires gradients through the discriminator even when discriminator parameters are held fixed.
- Looking only at a few attractive samples can hide mode collapse and memorization.
When to use it
Use a GAN when direct generation in one network pass is useful and you can evaluate quality and diversity carefully. Avoid choosing it solely for a low training loss, since adversarial losses are difficult to interpret as sample quality.