ConceptAI & ML
Markov decision processes
A generalization of search problems where you are not entirely sure what your actions are going to do until you try them.
Know first
Named as a prerequisite by the lectures that define this. The number is how many say so. A grey one has no page yet.
Opens up
Concepts that name this one as a prerequisite.
Lecturers
2Each in their own terms, from the easiest level.
David Silver
Markov decision processes formally describe an environment for reinforcement learning where the environment is fully observable.
Lectures that assume it
69The first 12 are shown.
- Model-Free Control — ε-Greedy SARSA & Off-Policy Q-Learning
- Planning by Dynamic Programming — Policy & Value Iteration
- Value Function Approximation & Deep Q-Networks (DQN)
- Policy Gradients I: Deriving the REINFORCE Estimator & Policy Objective
- Natural Policy Gradients, Covariant Updates & The Fisher Information Matrix
- Deep Reinforcement Learning
- Deep Reinforcement Learning
- Exploration of advanced topics in deep reinforcement learning
- RL Theory
- Reinforcement Learning Part 1/2
- Decision Transformer: Reinforcement Learning via Sequence Modeling
- Imitation Learning II: Covariate Shift & Dataset Aggregation (DAgger)
Connections
80The lectures that define it or assume it, and the concepts next to it.
Concepts9
- Action
- Agent and environment
- Search problem
- State
- Offline reinforcement learning
- Temporal difference learning
Show 3 moreShow fewer
Lectures71
- Markov Processes, Reward Processes & Bellman Equations
- Markov Decision Processes (MDPs) Part 1/2
- Model-Free Control — ε-Greedy SARSA & Off-Policy Q-Learning
- Planning by Dynamic Programming — Policy & Value Iteration
- Value Function Approximation & Deep Q-Networks (DQN)
- Policy Gradients I: Deriving the REINFORCE Estimator & Policy Objective
Show 65 moreShow fewer
- Natural Policy Gradients, Covariant Updates & The Fisher Information Matrix
- Deep Reinforcement Learning
- Deep Reinforcement Learning
- Exploration of advanced topics in deep reinforcement learning
- RL Theory
- Reinforcement Learning Part 1/2
- Decision Transformer: Reinforcement Learning via Sequence Modeling
- Imitation Learning II: Covariate Shift & Dataset Aggregation (DAgger)
- Exploration and Exploitation
- Decision Networks and Value of Perfect Information (VPI)
- How Can Robots Learn to Interact and Coordinate with Others?
- Exploration of fundamental concepts in deep reinforcement learning
- Deep Reinforcement Learning
- Inverse Reinforcement Learning, Part 2
- The Alberta Plan for AI Research: Continual Reinforcement Learning
- Classic Games
- Introduction to Reinforcement Learning (CS234, 2024, Lecture 1)
- Model-Free Prediction — Monte Carlo, TD(0) & TD(λ)
- Dynamic Programming I
- Markov Decision Processes (MDPs) Part 2/2
- Trust Region Policy Optimization (TRPO) & Monotonic Policy Improvement
- What Humans Want: Models of Preference and Reward in Minds and Machines
- Simulation-Driven Robot Learning for the Lazy Researcher
- Expanding What Robots Can Learn in the Real World
- Foundational concepts in deep reinforcement learning
- Exploration of deep reinforcement learning, likely
- Foundational concepts in deep reinforcement learning
- Model-Based Reinforcement Learning with Policies
- Foundational concepts in deep reinforcement learning
- Offline Reinforcement Learning
- Offline Reinforcement Learning (CS 285, Lecture 15, Part 3)
- Control as Inference, Part 1
- Inverse Reinforcement Learning, Part 1
- Inverse Reinforcement Learning, Part 4
- Challenges & Open Problems in Deep Reinforcement Learning
- LLM-Guided Reinforcement Learning for Mastery Learning: Large-Scale Field Evidence
- Model-Based Reinforcement Learning with Policies
- Policy Gradient Methods
- Integrating Learning and Planning
- Dynamic Programming III
- Stochastic Control
- Actor-Critic Methods: Value Function Bootstrapping & Advantage Estimation
- AI in Science and Engineering: Virtues and Vices of Robot Foundation Models
- Generative Control, Action Chunking, and Moravec’s Paradox
- Robotics, Language, and Vision
- Exploration of advanced topics in deep reinforcement learning
- Offline Reinforcement Learning
- Towards a Statistical Foundation for Reinforcement Learning
- RL with Sequence Models & Language Models, Part 1 (CS 285, Lecture 21)
- Transfer Learning & Meta-Learning
- Challenges and Open Problems in Deep Reinforcement Learning (CS 285, Lecture 23, Part 1)
- Imitation Learning III: Inverse RL & Generative Adversarial Imitation Learning (GAIL)
- Reinforcement Learning Part 2/2
- Deep Inference with Karl J. Friston
- Foundational concepts in deep reinforcement learning, likely
- Exploration of advanced topics in deep reinforcement learning
- Exploration of advanced topics in deep reinforcement learning
- Exploration of advanced topics in deep reinforcement learning
- RL Theory
- Guest Lecture: Aviral Kumar (UC Berkeley CS 285)
- Guest Lecture: Dorsa Sadigh (UC Berkeley CS 285)
- Imitation Learning, Foundation Models, and Course Wrap-up
- Sequential Optimization of Dynamically Augmented Conditional Value-at-Risk
- An Occam's Razor Principle for Transformers?
- Advanced Topics and Course Summary