Opening Exploration of advanced topics in deep reinforcement learning
Exploration of advanced topics in deep reinforcement learning
- Length
- 13 min
- Level
- Graduate
- Recorded
- UC Berkeley2020
Chapter 1 of 3: Stochastic linear quadratic regulators
Linear quadratic regulators extend to stochastic and nonlinear dynamics in this session of Levine's CS 285: Deep Reinforcement Learning at UC Berkeley. Levine first shows that Gaussian noise in linear dynamics leaves the optimal control law unchanged, though the trajectory now visits a distribution of states. For nonlinear systems, iterative LQR takes Taylor expansions of the dynamics and cost around the current trajectory and solves each local subproblem with the standard backward-forward recursion. Differential dynamic programming adds second-order dynamics terms, behaving much like Newton's method. A line search with a scaling parameter on the control deviation ensures each forward pass improves the cost and prevents overshooting.

Your comment is kept while you sign up.Sign in
Levine shows why Gaussian noise leaves the LQR control law unchanged, then builds iLQR for nonlinear systems.
Linear quadratic regulators extend to stochastic and nonlinear dynamics in this session of Levine's CS 285: Deep Reinforcement Learning at UC Berkeley. Levine first shows that Gaussian noise in linear dynamics leaves the optimal control law unchanged, though the trajectory now visits a distribution of states. For nonlinear systems, iterative LQR takes Taylor expansions of the dynamics and cost around the current trajectory and solves each local subproblem with the standard backward-forward recursion. Differential dynamic programming adds second-order dynamics terms, behaving much like Newton's method. A line search with a scaling parameter on the control deviation ensures each forward pass improves the cost and prevents overshooting.
This lecture
Lecture 31 of 73 of Deep Reinforcement Learning.
Choose a node to see how it connects to this lecture. What comes before is on the leftat the top. What comes after is on the rightat the bottom. The ring is what it teaches. People and places are above. Other lectures and books are below.
Press a chosen node again to open it.