ConceptAI & ML
Markov decision process
A mathematical framework for modeling decision making in situations where outcomes are partly random and partly under the control of a decision maker.
Know first
Named as a prerequisite by the lectures that define this. The number is how many say so. A grey one has no page yet.
Opens up
Concepts that name this one as a prerequisite.
Lecturers
10Each in their own terms, from the easiest level.
David Silver
The main formalism for reinforcement learning when working with fully observable environments.Brian Yu
A model for decision-making that represents states, actions, and their rewards.Pieter Abbeel
A mathematical framework for modeling decision-making in situations where outcomes are partly random and partly under the control of a decision maker.Fei-Fei Li, Justin Johnson & Serena Yeung
A mathematical formulation of the reinforcement learning problem that satisfies the Markov property.Onur Mutlu
A mathematical framework for modeling decision-making where outcomes are partly random and partly under the control of a decision-maker.Sergey Levine
A decision process defined by states, actions, rewards, and transition probabilities on a Markovian state.Saman Amarasinghe
A mathematical framework for modeling decision making where outcomes are partly random and partly under the control of a decision maker.Max Simchowitz
A formal framework for decision-making specified by states, actions, transition probabilities, rewards, an initial state distribution, and a problem horizon.Eugene A. Feinberg
A mathematical framework for modeling decision making in situations where outcomes are partly random and partly under the control of a decision maker, defined by states, actions, transition probabilities, and costs.
Lectures that assume it
15The first 12 are shown.
- Planning by Dynamic Programming — Policy & Value Iteration
- Reinforcement Learning Part 1/2
- Decision Transformer: Reinforcement Learning via Sequence Modeling
- Model-Free Prediction — Monte Carlo, TD(0) & TD(λ)
- Acrobots, Cart-poles, and Quadrotors I
- Foundational concepts in deep reinforcement learning
- LLM-Guided Reinforcement Learning for Mastery Learning: Large-Scale Field Evidence
- Policy Gradient Methods
- Integrating Learning and Planning
- AI in Science and Engineering: Virtues and Vices of Robot Foundation Models
- Towards a Statistical Foundation for Reinforcement Learning
- Transfer Learning & Meta-Learning
Connections
35The lectures that define it or assume it, and the concepts next to it.
Concepts9
Show 3 moreShow fewer
Lectures26
- Introduction to Reinforcement Learning
- Markov Processes, Reward Processes & Bellman Equations
- Harvard CS50’s Artificial Intelligence with Python – Full University Course
- Memory Controller Microarchitecture, Row Policies & Request Scheduling
- Introduction to Reinforcement Learning (CS234, 2024, Lecture 1)
- Policy Gradient Foundations: Trajectory Distributions & High-Dimensional State Spaces
Show 20 moreShow fewer
- Markov Decision Processes (MDPs) Part 2/2
- Compiler 2.0: Building the Next Generation of Compilers with Machine Learning
- Deep Reinforcement Learning
- Generative Control, Action Chunking, and Moravec’s Paradox
- Sequential Optimization of Dynamically Augmented Conditional Value-at-Risk
- Planning by Dynamic Programming — Policy & Value Iteration
- Reinforcement Learning Part 1/2
- Decision Transformer: Reinforcement Learning via Sequence Modeling
- Model-Free Prediction — Monte Carlo, TD(0) & TD(λ)
- Acrobots, Cart-poles, and Quadrotors I
- Foundational concepts in deep reinforcement learning
- LLM-Guided Reinforcement Learning for Mastery Learning: Large-Scale Field Evidence
- Policy Gradient Methods
- Integrating Learning and Planning
- AI in Science and Engineering: Virtues and Vices of Robot Foundation Models
- Towards a Statistical Foundation for Reinforcement Learning
- Transfer Learning & Meta-Learning
- Transfer Learning & Meta-Learning (CS 285, Lecture 22, Part 5)
- Output feedback
- Reinforcement Learning Part 2/2