VAE Objective: ELBO Gap, Gaussian KL and Posterior Collapse

MediumDeep Learning

Reported in interviews at Google

Problem statement

How would you explain the training objective of a variational autoencoder? An image model and a sentence model use Gaussian encoders, standard normal priors, and a lower bound on data likelihood. The team wants to understand the bound, calculate its penalty, and recognize when the decoder ignores the latent code.

Explain the likelihood bound, calculate Gaussian KL, and diagnose unused latent codes.

Show hint

What do you get if you write $\log p(x)$ as an expectation under the encoder and multiply and divide by $q(z \mid x)$ inside the log?

Relevant roles: ML Engineer, Research Scientist, Deep Learning Engineer and Applied Scientist.

Related free problems