Opening Control as Inference, Part 5
Control as Inference, Part 5
- Length
- 11 min
- Level
- Graduate
- Recorded
- UC Berkeley2020
Chapter 1 of 4: Soft Q-learning and exploration
Deep reinforcement learning can frame optimal control as probabilistic inference to build robust exploration strategies. The lecture explains soft Q-learning by tracking multiple hypotheses in an ant maze environment, avoiding the local optima traps of standard methods. It then contrasts a random expansion strategy with deterministic algorithms to demonstrate how pre-training with entropy maximization accelerates fine-tuning. Finally, it introduces soft actor-critic as an off-policy continuous control algorithm, showing its practical application in real-world robotic tasks like Lego block stacking and quadruped locomotion.
Your comment is kept while you sign up.Sign in
Derives soft Q-learning and soft actor-critic to demonstrate faster fine-tuning and robust real-world robotic control.
Deep reinforcement learning can frame optimal control as probabilistic inference to build robust exploration strategies. The lecture explains soft Q-learning by tracking multiple hypotheses in an ant maze environment, avoiding the local optima traps of standard methods. It then contrasts a random expansion strategy with deterministic algorithms to demonstrate how pre-training with entropy maximization accelerates fine-tuning. Finally, it introduces soft actor-critic as an off-policy continuous control algorithm, showing its practical application in real-world robotic tasks like Lego block stacking and quadruped locomotion.
Reinforcement Learning with Deep Energy-Based Policies
Soft Actor-Critic Algorithms and Applications
This lecture
Lecture 59 of 73 of Deep Reinforcement Learning.
Choose a node to see how it connects to this lecture. What comes before is on the leftat the top. What comes after is on the rightat the bottom. The ring is what it teaches. People and places are above. Other lectures and books are below.
Press a chosen node again to open it.