Opening Foundational concepts in deep reinforcement learning, likely
Foundational concepts in deep reinforcement learning, likely
- Length
- 14 min
- Level
- Undergraduate
- Recorded
- UC Berkeley2020
Chapter 1 of 3: Fitted Q-iteration and online Q-learning
Deep reinforcement learning utilizes fitted Q-iteration algorithms to handle value-based methods without explicit policy representation or transition probability knowledge. The lecture builds from general fitted Q-iteration principles to online Q-learning and explores the severe numerical limitations of proper residual gradient methods. It analyzes the correlation problem inherent in sequential state transitions and addresses how parallel workers and replay buffers mitigate this issue. The final section assembles a complete deep Q-learning algorithm that combines batched IID sampling from a replay buffer with periodic policy data collection.

Your comment is kept while you sign up.Sign in
Derives fitted Q-iteration algorithms and explains how replay buffers solve online Q-learning correlation problems.
Deep reinforcement learning utilizes fitted Q-iteration algorithms to handle value-based methods without explicit policy representation or transition probability knowledge. The lecture builds from general fitted Q-iteration principles to online Q-learning and explores the severe numerical limitations of proper residual gradient methods. It analyzes the correlation problem inherent in sequential state transitions and addresses how parallel workers and replay buffers mitigate this issue. The final section assembles a complete deep Q-learning algorithm that combines batched IID sampling from a replay buffer with periodic policy data collection.
This lecture
Lecture 20 of 73 of Deep Reinforcement Learning.
Choose a node to see how it connects to this lecture. What comes before is on the leftat the top. What comes after is on the rightat the bottom. The ring is what it teaches. People and places are above. Other lectures and books are below.
Press a chosen node again to open it.