Opening Model-Based Reinforcement Learning with Policies
Model-Based Reinforcement Learning with Policies
- Length
- 29 min
- Level
- Graduate
- Recorded
- UC Berkeley2022
Chapter 1 of 3: Successor representations
Successor representations predict the discounted distribution of future states under a policy, independent of any reward function. Levine derives them from policy evaluation, following Dayan (1993), and shows that a value function becomes an inner product between the successor representation and a reward vector. Successor features, after Barreto et al., project these representations onto a feature basis so they scale to large state spaces. For continuous states, C-Learning trains a binary classifier to estimate the successor representation. The session, from UC Berkeley's CS 285, assumes MDPs, value functions and basic linear algebra, and notes that these methods support policy iteration rather than direct optimal control.

Your comment is kept while you sign up.Sign in
Levine derives successor representations from policy evaluation and recasts their estimation as binary classification.
Successor representations predict the discounted distribution of future states under a policy, independent of any reward function. Levine derives them from policy evaluation, following Dayan (1993), and shows that a value function becomes an inner product between the successor representation and a reward vector. Successor features, after Barreto et al., project these representations onto a feature basis so they scale to large state spaces. For continuous states, C-Learning trains a binary classifier to estimate the successor representation. The session, from UC Berkeley's CS 285, assumes MDPs, value functions and basic linear algebra, and notes that these methods support policy iteration rather than direct optimal control.
Improving Generalisation for Temporal Difference Learning: The Successor Representation
Improving Generalisation for Temporal Difference Learning
Successor Features for Transfer in Reinforcement Learning
C-Learning: Learning to Achieve Goals via Recursive Classification
This lecture
Lecture 38 of 73 of Deep Reinforcement Learning.
Choose a node to see how it connects to this lecture. What comes before is on the leftat the top. What comes after is on the rightat the bottom. The ring is what it teaches. People and places are above. Other lectures and books are below.
Press a chosen node again to open it.