ConceptAI & ML
Markov property
The property where the future state is conditionally independent of past states given the current state.
Know first
Named as a prerequisite by the lectures that define this. The number is how many say so. A grey one has no page yet.
Opens up
Concepts that name this one as a prerequisite.
Absorbing state1Chapman-Kolmogorov equation (multistep transition probability)1Continuous-state Markov process1Hidden Markov model1History states1Homogeneous Markov measure1Independent process noise1Markov decision process1Markov process1Partial observability in policy gradients1Policy gradients under partial observability1State1
Lecturers
6Each in their own terms, from the easiest level.
David Silver
A state is Markov if and only if the future is independent of the past given the present.Sergey Levine
Pieter Abbeel
The property that the future state depends only on the current state, not on the past.Joe Blitzstein
A conditional probability statement relating the present state to the future state, helping determine what will happen at a future time step given the current state.Fei-Fei Li, Justin Johnson & Serena Yeung
The property that the current state completely characterizes the state of the world.Alberto Abbondandolo
The future state of a system depends only upon the present state, not on the sequence of events that preceded it.
Lectures that assume it
7Connections
26The lectures that define it or assume it, and the concepts next to it.
Concepts10
Lectures16
- Introduction to Reinforcement Learning
- Markov Processes, Reward Processes & Bellman Equations
- Hidden Markov Models
- Markov Chains
- Gibbs Measures on Infinite Trees and Homoclinic Points
- Policy Gradient Foundations: Trajectory Distributions & High-Dimensional State Spaces
Show 10 moreShow fewer
- Imitation Learning I: Behavioral Cloning & Distributional Shift
- Deep Reinforcement Learning
- RL with Sequence Models & Language Models, Part 1 (CS 285, Lecture 21)
- Gambler's Ruin and Random Variables
- Markov Decision Processes (MDPs) Part 2/2
- Convex Optimization II: Stochastic Model Predictive Control and Branch-and-Bound Methods
- Stochastic dynamics
- Particle Filtering
- Robotics, Language, and Vision
- Exploration of fundamental concepts in deep reinforcement learning