Opening Exploration of deep reinforcement learning algorithms
Exploration of deep reinforcement learning algorithms
- Length
- 17 min
- Level
- Undergraduate
- Recorded
- UC Berkeley2020
Chapter 1 of 3: Uncertainty in dynamics models
Uncertainty-aware dynamics models keep a model-based reinforcement learning optimizer from exploiting erroneous out-of-distribution predictions. Levine first shows that the entropy of a model's output fails, because maximum likelihood training captures aleatoric noise rather than epistemic uncertainty about parameters. Bayesian neural networks fix this by keeping a distribution over weights and integrating parameters out, with Blundell et al. and Gal et al.'s Concrete Dropout cited for methods. Bootstrap ensembles then approximate the parameter posterior as a mixture of Dirac delta distributions. Levine notes that resampling with replacement is often unnecessary, since random initialization and stochastic gradient descent already diversify the networks.

Your comment is kept while you sign up.Sign in
Shows why output entropy misses epistemic uncertainty in learned dynamics models, then builds bootstrap ensembles.
Uncertainty-aware dynamics models keep a model-based reinforcement learning optimizer from exploiting erroneous out-of-distribution predictions. Levine first shows that the entropy of a model's output fails, because maximum likelihood training captures aleatoric noise rather than epistemic uncertainty about parameters. Bayesian neural networks fix this by keeping a distribution over weights and integrating parameters out, with Blundell et al. and Gal et al.'s Concrete Dropout cited for methods. Bootstrap ensembles then approximate the parameter posterior as a mixture of Dirac delta distributions. Levine notes that resampling with replacement is often unnecessary, since random initialization and stochastic gradient descent already diversify the networks.
Weight Uncertainty in Neural Networks
Concrete Dropout
This lecture
Lecture 34 of 73 of Deep Reinforcement Learning.
Choose a node to see how it connects to this lecture. What comes before is on the leftat the top. What comes after is on the rightat the bottom. The ring is what it teaches. People and places are above. Other lectures and books are below.
Press a chosen node again to open it.