Opening Exploration of advanced topics in deep reinforcement learning
Exploration of advanced topics in deep reinforcement learning
- Length
- 15 min
- Level
- Graduate
- Recorded
- UC Berkeley2020
Chapter 1 of 3: Unsupervised goal-reaching exploration
Unsupervised exploration in deep reinforcement learning can be framed as an agent proposing goals and practicing reaching them, with no reward function. In this session of UC Berkeley's CS 285, Levine starts from a variational autoencoder that maps image observations into a latent space and samples goal images. A goal-conditioned policy then attempts those goals and collects new data. Skew-Fit refits the generative model by weighted maximum likelihood, raising past densities to a negative exponent so rarely seen states gain probability. Levine shows that the procedure maximizes mutual information between states and goals, then closes with robot manipulators that learn door opening without supervision.

Your comment is kept while you sign up.Sign in
Levine builds Skew-Fit, upweighting rare states so a robot proposes its own goals and learns to open doors.
Unsupervised exploration in deep reinforcement learning can be framed as an agent proposing goals and practicing reaching them, with no reward function. In this session of UC Berkeley's CS 285, Levine starts from a variational autoencoder that maps image observations into a latent space and samples goal images. A goal-conditioned policy then attempts those goals and collects new data. Skew-Fit refits the generative model by weighted maximum likelihood, raising past densities to a negative exponent so rarely seen states gain probability. Levine shows that the procedure maximizes mutual information between states and goals, then closes with robot manipulators that learn door opening without supervision.
Visual Reinforcement Learning with Imagined Goals
Skew-Fit: State-Covering Self-Supervised Reinforcement Learning
Auto-Encoding Variational Bayes
This lecture
Lecture 43 of 73 of Deep Reinforcement Learning.
Choose a node to see how it connects to this lecture. What comes before is on the leftat the top. What comes after is on the rightat the bottom. The ring is what it teaches. People and places are above. Other lectures and books are below.
Press a chosen node again to open it.