Opening Exploration of advanced topics in deep reinforcement learning
Exploration of advanced topics in deep reinforcement learning
- Length
- 16 min
- Level
- Undergraduate
- Recorded
- UC Berkeley2020
Chapter 1 of 3: Optimistic exploration
Deep reinforcement learning methods rely on principled exploration strategies derived from multi-armed bandit settings to minimize regret. Optimistic exploration adds a variance-based bonus to reward estimates, incentivizing actions with high uncertainty. Posterior sampling maintains a belief distribution over models, sampling parameters to guide action selection. Information-directed sampling balances expected suboptimality against expected information gain to determine optimal actions. These tractable foundational strategies provide the theoretical basis for exploring more complex Markov decision processes.

Your comment is kept while you sign up.Sign in
Levine compares optimism, posterior sampling and information-directed sampling as bandit routes to low regret.
Deep reinforcement learning methods rely on principled exploration strategies derived from multi-armed bandit settings to minimize regret. Optimistic exploration adds a variance-based bonus to reward estimates, incentivizing actions with high uncertainty. Posterior sampling maintains a belief distribution over models, sampling parameters to guide action selection. Information-directed sampling balances expected suboptimality against expected information gain to determine optimal actions. These tractable foundational strategies provide the theoretical basis for exploring more complex Markov decision processes.
Finite-time analysis of the multiarmed bandit problem
An Empirical Evaluation of Thompson Sampling
Learning to Optimize via Information-Directed Sampling
This lecture
Lecture 40 of 73 of Deep Reinforcement Learning.
Choose a node to see how it connects to this lecture. What comes before is on the leftat the top. What comes after is on the rightat the bottom. The ring is what it teaches. People and places are above. Other lectures and books are below.
Press a chosen node again to open it.