Endpoint Replay: Compressing the Recency Buffer in Deep Reinforcement Learning
PhD student, University of Alberta
Introducing Endpoint Replay, an algorithm that significantly compresses recency buffers in deep reinforcement learning by pairing a recency buffer with a coreset of chained n-step transitions.
In this seminar from the Alberta Machine Intelligence Institute and the Department of Computing Science, Parham Panahi, a PhD student at the University of Alberta, explains why sub-selecting experience to shrink replay buffers leads to poor bootstrap value estimates, introduces Endpoint Replay: an algorithm pairing a recency buffer with a coreset of chained n-step transitions updated with Expectile Sarsa, matching standard-buffer performance with 10x-50x less storage and outperforming equal-sized buffers, reservoir-sampling coresets, and MeDQN.
BIO: Parham Panahi is a PhD student at the University of Alberta working on deep reinforcement learning, generative models, and continual learning, advised by Adam White and Michael Bowling. A unifying theme of his research is how intelligent agents store, use, and learn from their past experience.