Opening Exploration of advanced topics in deep reinforcement learning
Exploration of advanced topics in deep reinforcement learning
- Length
- 10 min
- Level
- Undergraduate
- Recorded
- UC Berkeley2020
Chapter 1 of 4: Before the lecture
Deep reinforcement learning extends Q-learning to continuous action spaces through three distinct methodological choices. The lecture examines derivative-free stochastic optimization methods like the cross-entropy method for approximating the maximum over actions. It then presents normalized advantage functions, which restrict the Q-function architecture to a quadratic form ensuring closed-form maximization. Finally, it details deep deterministic policy gradient algorithms, which train a second neural network to learn an approximate maximizer using the chain rule.

Your comment is kept while you sign up.Sign in
Extends Q-learning to continuous action spaces using random sampling and quadratic function classes.
Deep reinforcement learning extends Q-learning to continuous action spaces through three distinct methodological choices. The lecture examines derivative-free stochastic optimization methods like the cross-entropy method for approximating the maximum over actions. It then presents normalized advantage functions, which restrict the Q-function architecture to a quadratic form ensuring closed-form maximization. Finally, it details deep deterministic policy gradient algorithms, which train a second neural network to learn an approximate maximizer using the chain rule.
Continuous Deep Q-Learning with Model-Based Acceleration
Continuous control with deep reinforcement learning
This lecture
Lecture 23 of 73 of Deep Reinforcement Learning.
Choose a node to see how it connects to this lecture. What comes before is on the leftat the top. What comes after is on the rightat the bottom. The ring is what it teaches. People and places are above. Other lectures and books are below.
Press a chosen node again to open it.