Opening Exploration of advanced topics in deep reinforcement learning
Exploration of advanced topics in deep reinforcement learning
- Length
- 15 min
- Level
- Undergraduate
- Recorded
- UC Berkeley2020
Chapter 1 of 2: Count-based exploration and its limitations
Count-based exploration in deep reinforcement learning extends bandit algorithms to Markov decision processes by adding a visitation bonus to the reward function. Because exact state visitation counts become useless in large or continuous spaces, density models compute pseudo-counts to approximate how often states are visited. The lecture compares optimistic exploration, Thompson sampling, and information gain methods before deriving the pseudo-count formulation from density ratio updates. It concludes by examining how simple models like the context-tree switching model enable exploration in games such as Montezuma's Revenge.
Your comment is kept while you sign up.Sign in
Derives pseudo-count exploration bonuses from density model updates to solve continuous reinforcement learning problems.
Count-based exploration in deep reinforcement learning extends bandit algorithms to Markov decision processes by adding a visitation bonus to the reward function. Because exact state visitation counts become useless in large or continuous spaces, density models compute pseudo-counts to approximate how often states are visited. The lecture compares optimistic exploration, Thompson sampling, and information gain methods before deriving the pseudo-count formulation from density ratio updates. It concludes by examining how simple models like the context-tree switching model enable exploration in games such as Montezuma's Revenge.
Unifying Count-Based Exploration and Intrinsic Motivation
An Analysis of Model-Based Interval Estimation for Markov Decision Processes
Near-Bayesian Exploration in Reinforcement Learning
This lecture
Lecture 41 of 73 of Deep Reinforcement Learning.
Choose a node to see how it connects to this lecture. What comes before is on the leftat the top. What comes after is on the rightat the bottom. The ring is what it teaches. People and places are above. Other lectures and books are below.
Press a chosen node again to open it.