Opening Deep Reinforcement Learning
Deep Reinforcement Learning
- Length
- 14 min
- Level
- Graduate
- Recorded
- UC Berkeley2020
Chapter 1 of 4: Before the lecture
Unsupervised reinforcement learning asks how an agent can acquire diverse skills with no reward function at all. Levine, in a short session of UC Berkeley CS 285, recasts exploration from coping with sparse rewards to building a repertoire of reusable behaviours. He reviews entropy as a measure of how broad a policy's state distribution is. Mutual information follows, written as a KL divergence between a joint distribution and the product of its marginals. Empowerment, credited to Daniel Polani and colleagues, then becomes the mutual information between current action and next state: high entropy of future states, low uncertainty given the action.

Your comment is kept while you sign up.Sign in
Levine writes empowerment as the mutual information between action and next state, a measure of an agent's control.
Unsupervised reinforcement learning asks how an agent can acquire diverse skills with no reward function at all. Levine, in a short session of UC Berkeley CS 285, recasts exploration from coping with sparse rewards to building a repertoire of reusable behaviours. He reviews entropy as a measure of how broad a policy's state distribution is. Mutual information follows, written as a KL divergence between a joint distribution and the product of its marginals. Empowerment, credited to Daniel Polani and colleagues, then becomes the mutual information between current action and next state: high entropy of future states, low uncertainty given the action.
This lecture
Lecture 42 of 73 of Deep Reinforcement Learning.
Choose a node to see how it connects to this lecture. What comes before is on the leftat the top. What comes after is on the rightat the bottom. The ring is what it teaches. People and places are above. Other lectures and books are below.
Press a chosen node again to open it.