ConceptAI & ML
Policy gradient
A class of reinforcement learning algorithms that optimize the policy parameters directly by maximizing the expected cumulative reward.
Know first
Named as a prerequisite by the lectures that define this. The number is how many say so. A grey one has no page yet.
Reinforcement learning3Gradient ascent2Gradient descent2Expectation1Expected reward1Expected reward objective1Log probabilities1Policy1
Opens up
Concepts that name this one as a prerequisite.
Natural policy gradient3REINFORCE algorithm3Actor-critic algorithm2Baseline2Action-value actor-critic1Baseline function1Batch reinforcement learning via importance sampling1Continual learning via temporal consensus1Covariant/natural policy gradient1Discount factors for policy gradients1DPO gradient mechanics1Finite difference policy gradient1
Lecturers
4Each in their own terms, from the easiest level.
Sergey Levine
An expression used to estimate the gradient of expected reward with respect to policy parameters, often used as a model-free reinforcement learning algorithm.David Silver
A vector of partial derivatives of the policy objective function with respect to the policy parameters, used to ascend to a local maximum.Jacob Andreas
A reinforcement learning method used to optimize the language model parameters to maximize expected reward.
Lectures that assume it
16The first 12 are shown.
- Natural Policy Gradients, Covariant Updates & The Fisher Information Matrix
- Deep Reinforcement Learning
- RL with Sequence Models & Language Models, Part 2
- Policy Gradients II: Variance Reduction, Baselines & Reward-to-Go Causality
- How Can Robots Learn to Interact and Coordinate with Others?
- Trust Region Policy Optimization (TRPO) & Monotonic Policy Improvement
- Expanding What Robots Can Learn in the Real World
- Exploration of advanced topics in deep reinforcement learning
- Offline Reinforcement Learning
- Reinforcement Learning from Human Feedback: Algorithms & Applications
- Actor-Critic Methods: Value Function Bootstrapping & Advantage Estimation
- Exploration of advanced topics in deep reinforcement learning
Connections
32The lectures that define it or assume it, and the concepts next to it.
Concepts9
- Reinforcement learning
- Gradient ascent
- Gradient descent
- Expectation
- Natural policy gradient
- REINFORCE algorithm
Show 3 moreShow fewer
Lectures23
- Policy Gradients I: Deriving the REINFORCE Estimator & Policy Objective
- Variational Inference, Part 3
- Policy Gradient Methods
- Deep Reinforcement Learning
- Model-Based Reinforcement Learning with Policies
- Language Models: Prompting, Chain-of-Thought, and Instruction-Tuning
Show 17 moreShow fewer
- Exploration of deep reinforcement learning architectures and algorithms
- Natural Policy Gradients, Covariant Updates & The Fisher Information Matrix
- Deep Reinforcement Learning
- RL with Sequence Models & Language Models, Part 2
- Policy Gradients II: Variance Reduction, Baselines & Reward-to-Go Causality
- How Can Robots Learn to Interact and Coordinate with Others?
- Trust Region Policy Optimization (TRPO) & Monotonic Policy Improvement
- Expanding What Robots Can Learn in the Real World
- Exploration of advanced topics in deep reinforcement learning
- Offline Reinforcement Learning
- Reinforcement Learning from Human Feedback: Algorithms & Applications
- Actor-Critic Methods: Value Function Bootstrapping & Advantage Estimation
- Exploration of advanced topics in deep reinforcement learning
- Offline Reinforcement Learning
- RL with Sequence Models & Language Models, Part 1 (CS 285, Lecture 21)
- Challenges and Open Problems in Deep Reinforcement Learning (CS 285, Lecture 23, Part 1)
- Exploration of fundamental concepts in deep reinforcement learning