Opening Exploration of advanced topics in deep reinforcement learning
Exploration of advanced topics in deep reinforcement learning
- Length
- 23 min
- Level
- Undergraduate
- Recorded
- UC Berkeley2020
Chapter 1 of 2: Stochastic optimization methods for planning
Levine refines random shooting into the cross-entropy method, then turns to Monte Carlo tree search and its UCT rule.
Planning with a known dynamics model can be cast as black-box optimization over sequences of actions. Levine begins with random shooting, which samples many action sequences in parallel and keeps the best one. The cross-entropy method improves on it by repeatedly refitting the sampling distribution to the elite samples. Both methods ignore temporal structure and plan open-loop, so Levine turns to Monte Carlo tree search for closed-loop feedback, estimating state values from rollouts. The UCT tree policy weighs average reward against visit counts, and the session from UC Berkeley's CS 285 assumes familiarity with Markov decision processes.
Planning with a known dynamics model can be cast as black-box optimization over sequences of actions. Levine begins with random shooting, which samples many action sequences in parallel and keeps the best one. The cross-entropy method improves on it by repeatedly refitting the sampling distribution to the elite samples. Both methods ignore temporal structure and plan open-loop, so Levine turns to Monte Carlo tree search for closed-loop feedback, estimating state values from rollouts. The UCT tree policy weighs average reward against visit counts, and the session from UC Berkeley's CS 285 assumes familiarity with Markov decision processes.
A Survey of Monte Carlo Tree Search Methods

This lecture
Lecture 29 of 73 of Deep Reinforcement Learning.
Choose a node to see how it connects to this lecture. What comes before is on the leftat the top. What comes after is on the rightat the bottom. The ring is what it teaches. People and places are above. Other lectures and books are below.
Press a chosen node again to open it.
Your comment is kept while you sign up.Sign in