Opening Variational Inference, Part 2
Variational Inference, Part 2
- Length
- 20 min
- Level
- Graduate
- Recorded
- UC Berkeley2020
Chapter 1 of 3: Variational inference framework
Variational inference replaces an intractable posterior over latent variables with a simple parameterized distribution, such as a Gaussian, and maximizes a lower bound on . In this part of Berkeley's CS 285, Levine applies Jensen's inequality to extract the evidence lower bound. Using the entropy term, he shows that equals the ELBO plus the KL divergence from the approximation to the true posterior. Maximizing the bound then alternates between improving the generative model and tightening the approximate distribution, both implemented as stochastic gradient steps. The cost of separate variational parameters for every data point motivates amortized inference with a neural network.
Your comment is kept while you sign up.Sign in
Levine derives the evidence lower bound via Jensen's inequality; its gap equals the KL divergence to the posterior.
Variational inference replaces an intractable posterior over latent variables with a simple parameterized distribution, such as a Gaussian, and maximizes a lower bound on . In this part of Berkeley's CS 285, Levine applies Jensen's inequality to extract the evidence lower bound. Using the entropy term, he shows that equals the ELBO plus the KL divergence from the approximation to the true posterior. Maximizing the bound then alternates between improving the generative model and tightening the approximate distribution, both implemented as stochastic gradient steps. The cost of separate variational parameters for every data point motivates amortized inference with a neural network.
This lecture
Lecture 53 of 73 of Deep Reinforcement Learning.
Choose a node to see how it connects to this lecture. What comes before is on the leftat the top. What comes after is on the rightat the bottom. The ring is what it teaches. People and places are above. Other lectures and books are below.
Press a chosen node again to open it.