Opening Exploration of deep reinforcement learning techniques
Exploration of deep reinforcement learning techniques
- Length
- 12 min
- Level
- Undergraduate
- Recorded
- UC Berkeley2020
Chapter 1 of 3: The moving target problem
Deep Q-learning constructs stable updates by combining replay buffers with a separate target network. Standard Q-learning resembles regression against a constantly moving target, so updating network parameters directly leads to divergence. The algorithm holds target parameters static over many steps before updating them from the primary network. This special case yields the classic Deep Q-Network algorithm used in practical implementations. Polyak averaging offers a smooth alternative to periodic hard updates for target parameters.
Your comment is kept while you sign up.Sign in
Derives the deep Q-learning algorithm and target network updates to stabilize value function regression.
Deep Q-learning constructs stable updates by combining replay buffers with a separate target network. Standard Q-learning resembles regression against a constantly moving target, so updating network parameters directly leads to divergence. The algorithm holds target parameters static over many steps before updating them from the primary network. This special case yields the classic Deep Q-Network algorithm used in practical implementations. Polyak averaging offers a smooth alternative to periodic hard updates for target parameters.
Mnih et al. '13
This lecture
Lecture 21 of 73 of Deep Reinforcement Learning.
Choose a node to see how it connects to this lecture. What comes before is on the leftat the top. What comes after is on the rightat the bottom. The ring is what it teaches. People and places are above. Other lectures and books are below.
Press a chosen node again to open it.