Opening Transfer Learning & Meta-Learning
Transfer Learning & Meta-Learning
- Length
- 44 min
- Level
- Undergraduate
- Recorded
- UC Berkeley2022
Chapter 1 of 6: Transfer learning and motivation
Levine builds domain adversarial networks with gradient reversal, then turns to domain randomization for sim-to-real.
Transfer learning in deep reinforcement learning uses experience from source MDPs to solve a new target MDP faster and better. Levine, teaching CS 285 at UC Berkeley, opens with Montezuma's Revenge and explains why pre-training and fine-tuning is harder in RL than in supervised learning. He states the invariance assumption and shows how a domain classifier with gradient reversal learns domain-invariant representations. Domain randomization follows, with EPOpt, CAD2RL, dynamics randomization and quadrupedal locomotion as cases, alongside off-dynamics RL via domain classifiers. The session closes by framing multi-task RL as a joint MDP solved with contextual and goal-conditioned policies, assuming Q-learning and policy gradients.
Transfer learning in deep reinforcement learning uses experience from source MDPs to solve a new target MDP faster and better. Levine, teaching CS 285 at UC Berkeley, opens with Montezuma's Revenge and explains why pre-training and fine-tuning is harder in RL than in supervised learning. He states the invariance assumption and shows how a domain classifier with gradient reversal learns domain-invariant representations. Domain randomization follows, with EPOpt, CAD2RL, dynamics randomization and quadrupedal locomotion as cases, alongside off-dynamics RL via domain classifiers. The session closes by framing multi-task RL as a joint MDP solved with contextual and goal-conditioned policies, assuming Q-learning and policy gradients.
Off-Dynamics Reinforcement Learning: Training for Transfer with Domain Classifiers
EPOpt: Learning Robust Neural Network Policies Using Model Ensembles

This lecture
Lecture 68 of 73 of Deep Reinforcement Learning.
Choose a node to see how it connects to this lecture. What comes before is on the leftat the top. What comes after is on the rightat the bottom. The ring is what it teaches. People and places are above. Other lectures and books are below.
Press a chosen node again to open it.
Your comment is kept while you sign up.Sign in
CAD2RL: Real Single-Image Flight without a Single Real Image
Sim-to-Real Transfer of Robotic Control with Dynamics Randomization
Learning Quadrupedal Locomotion over Challenging Terrain
Learning to achieve goals
Universal value function approximators
Hindsight experience replay
C-Learning: Learning to achieve goals via recursive classification