Opening Control as Inference, Part 1
Control as Inference, Part 1
- Length
- 21 min
- Level
- Graduate
- Recorded
- UC Berkeley2020
Chapter 1 of 4: Before the lecture
Control as inference reframes optimal control and reinforcement learning as probabilistic inference within a graphical model. Standard optimal control yields deterministic policies, yet humans and animals naturally exhibit stochastic behavior when completing tasks. By adding binary optimality variables whose probabilities scale with exponentiated negative rewards, the framework models near-optimal trajectories where mistakes in unimportant dimensions incur exponentially less penalty. This formulation allows message passing algorithms to compute policies, state occupancies, and backward messages.
Your comment is kept while you sign up.Sign in
Reframes optimal control as probabilistic inference to model stochastic, near-optimal behavior in agents.
Control as inference reframes optimal control and reinforcement learning as probabilistic inference within a graphical model. Standard optimal control yields deterministic policies, yet humans and animals naturally exhibit stochastic behavior when completing tasks. By adding binary optimality variables whose probabilities scale with exponentiated negative rewards, the framework models near-optimal trajectories where mistakes in unimportant dimensions incur exponentially less penalty. This formulation allows message passing algorithms to compute policies, state occupancies, and backward messages.
This lecture
Lecture 56 of 73 of Deep Reinforcement Learning.
Choose a node to see how it connects to this lecture. What comes before is on the leftat the top. What comes after is on the rightat the bottom. The ring is what it teaches. People and places are above. Other lectures and books are below.
Press a chosen node again to open it.