Opening Transfer Learning & Meta-Learning (CS 285, Lecture 22, Part 5)
Transfer Learning & Meta-Learning (CS 285, Lecture 22, Part 5)
- Length
- 13 min
- Level
- Graduate
- Recorded
- UC Berkeley2022
Chapter 1 of 3: Meta-reinforcement learning as a POMDP
Transfer reinforcement learning unifies meta-learning methods by framing them as partially observable Markov decision processes where an unknown latent variable determines the task. Levine models this inference problem using variational autoencoders to estimate task representations from historical states, actions, and rewards. This formulation enables exploration through posterior sampling, as implemented in the Pearl algorithm where transitions are averaged to parameterize latent distributions. The lecture contrasts this inference-based view with black-box recurrent models and gradient-based adaptation, highlighting trade-offs in meta-overfitting and optimization. Finally, it notes how meta-reinforcement learning can produce emergent behaviors like episodic recall and model-based adaptation.

Your comment is kept while you sign up.Sign in
Frames meta-reinforcement learning as a partially observed Markov decision process solved via variational inference.
Transfer reinforcement learning unifies meta-learning methods by framing them as partially observable Markov decision processes where an unknown latent variable determines the task. Levine models this inference problem using variational autoencoders to estimate task representations from historical states, actions, and rewards. This formulation enables exploration through posterior sampling, as implemented in the Pearl algorithm where transitions are averaged to parameterize latent distributions. The lecture contrasts this inference-based view with black-box recurrent models and gradient-based adaptation, highlighting trade-offs in meta-overfitting and optimization. Finally, it notes how meta-reinforcement learning can produce emergent behaviors like episodic recall and model-based adaptation.
Efficient Off-Policy Meta-Reinforcement Learning via Probabilistic Context Variables
Variational Task Embeddings for Fast Adaptation in Deep Reinforcement Learning
Meta reinforcement learning as task inference
Been There, Done That: Meta-Learning with Episodic Recall
Prefrontal Cortex as a Meta-Reinforcement Learning System
Causal Reasoning from Meta-Reinforcement Learning
This lecture
Lecture 69 of 73 of Deep Reinforcement Learning.
Choose a node to see how it connects to this lecture. What comes before is on the leftat the top. What comes after is on the rightat the bottom. The ring is what it teaches. People and places are above. Other lectures and books are below.
Press a chosen node again to open it.