Opening Inverse Reinforcement Learning, Part 4
Inverse Reinforcement Learning, Part 4
- Length
- 14 min
- Level
- Graduate
- Recorded
- UC Berkeley2020
Chapter 1 of 2: Connection between inverse reinforcement learning and generative
Inverse reinforcement learning can be written as a game between a policy and a reward-based discriminator, the same structure as a generative adversarial network. Levine builds the link from the Bayes optimal discriminator, which equals a density ratio between expert trajectories and policy trajectories weighted by an exponentiated reward. Following Finn, Christiano et al., this ties guided cost learning to GANs and energy-based models. Two algorithms follow: adversarial inverse reinforcement learning (Fu et al.) separates the reward from the dynamics so it transfers to new conditions. Generative adversarial imitation learning (Ho and Ermon) uses a plain binary classifier and recovers only the policy, and simulated locomotion tasks illustrate both. The session assumes basic reinforcement learning, Markov decision processes, neural network classifiers and density ratios.

Your comment is kept while you sign up.Sign in
Derives the Bayes optimal discriminator as a density ratio with an exponentiated reward, casting inverse RL as a GAN.
Inverse reinforcement learning can be written as a game between a policy and a reward-based discriminator, the same structure as a generative adversarial network. Levine builds the link from the Bayes optimal discriminator, which equals a density ratio between expert trajectories and policy trajectories weighted by an exponentiated reward. Following Finn, Christiano et al., this ties guided cost learning to GANs and energy-based models. Two algorithms follow: adversarial inverse reinforcement learning (Fu et al.) separates the reward from the dynamics so it transfers to new conditions. Generative adversarial imitation learning (Ho and Ermon) uses a plain binary classifier and recovers only the policy, and simulated locomotion tasks illustrate both. The session assumes basic reinforcement learning, Markov decision processes, neural network classifiers and density ratios.
Generative Adversarial Nets
Unpaired Image-to-Image Translation using Cycle-Consistent Adversarial Networks
Wasserstein GAN
Image-to-Image Translation with Conditional Adversarial Networks
A Connection Between Generative Adversarial Networks, Inverse Reinforcement Learning, and Energy-Based Models
Learning Robust Rewards with Adversarial Inverse Reinforcement Learning
Generative Adversarial Imitation Learning
Guided Cost Learning
Apprenticeship Learning via Inverse Reinforcement Learning
Maximum Entropy Inverse Reinforcement Learning
Deep Maximum Entropy Inverse Reinforcement Learning
This lecture
Lecture 62 of 73 of Deep Reinforcement Learning.
Choose a node to see how it connects to this lecture. What comes before is on the leftat the top. What comes after is on the rightat the bottom. The ring is what it teaches. People and places are above. Other lectures and books are below.
Press a chosen node again to open it.