Opening Foundational concepts in deep reinforcement learning
Foundational concepts in deep reinforcement learning
- Length
- 20 min
- Level
- Graduate
- Recorded
- UC Berkeley2020
Chapter 1 of 4: Before the lecture
Deep reinforcement learning struggles on tasks with sparse rewards because the agent must explore through trial and error before receiving feedback. This lecture examines the exploration-exploitation dilemma using multi-armed bandits as a model organism and formalizes them as partially observable Markov decision processes. It defines the Bayes-optimal strategy and measures the quality of alternative algorithms through cumulative regret over time. The discussion contrasts theoretically tractable bandit problems with the intractable large infinite Markov decision processes common in deep reinforcement learning.

Your comment is kept while you sign up.Sign in
Levine models exploration with multi-armed bandits and measures each strategy by its cumulative regret.
Deep reinforcement learning struggles on tasks with sparse rewards because the agent must explore through trial and error before receiving feedback. This lecture examines the exploration-exploitation dilemma using multi-armed bandits as a model organism and formalizes them as partially observable Markov decision processes. It defines the Bayes-optimal strategy and measures the quality of alternative algorithms through cumulative regret over time. The discussion contrasts theoretically tractable bandit problems with the intractable large infinite Markov decision processes common in deep reinforcement learning.
This lecture
Lecture 39 of 73 of Deep Reinforcement Learning.
Choose a node to see how it connects to this lecture. What comes before is on the leftat the top. What comes after is on the rightat the bottom. The ring is what it teaches. People and places are above. Other lectures and books are below.
Press a chosen node again to open it.