ConceptAI & ML
Markov chain
A stochastic process where random variables evolve over time in a way that is one step beyond IID, representing the state of a system moving randomly from state to state.
Know first
Named as a prerequisite by the lectures that define this. The number is how many say so. A grey one has no page yet.
Opens up
Concepts that name this one as a prerequisite.
Lecturers
7Each in their own terms, from the easiest level.
Joe Blitzstein
Brian Yu
A sequence of random variables where the distribution of each variable follows the Markov assumption.Peter Kempthorne, Jake Xia & Vasily Strela
A stochastic process whose effect of the past on the future is summarized only by the current state.Emma Brunskill
A memoryless random process consisting of a finite set of states and a transition model.Pieter Abbeel
The simplest instantiation of a random variable evolving over time where a variable replicates at every time step.Sergey Levine
A mathematical system consisting of a set of states and a transition function.Stephen Boyd
A stochastic process where a probability distribution evolves in time by being multiplied by a state transition matrix P.
Lectures that assume it
19The first 12 are shown.
- Markov Chains Continued Further (Statistics 110, Lecture 33)
- Bayes' Nets: Sampling
- Hidden Markov Models
- Complexity and Computability with Closed Timelike Curves
- Gibbs Measures on Infinite Trees and Homoclinic Points
- Controlling Stochastic Biophysical Processes, from Protein Folding to Evolution
- Privacy Amplification for Correlated-Noise Mechanisms via b-Min-Sep Subsampling
- Convex Optimization II: Localization and Cutting-Plane Methods
- Ross Recovery Theorem
- Leveraging Per-Instance Privacy for Machine Unlearning
- LTFs and Noise Stability
- Stochastic dynamics
Connections
38The lectures that define it or assume it, and the concepts next to it.
Concepts10
- Probability
- Random variable
- Probability distribution
- Conditional probability
- Conditional independence
- Stationary distribution
Show 4 moreShow fewer
Lectures28
- Markov Chains Continued: Irreducibility, Recurrence, Transience, and Reversibility (Statistics 110, Lecture 32)
- Harvard CS50’s Artificial Intelligence with Python – Full University Course
- Stochastic Processes I
- Markov Chains
- Introduction to Reinforcement Learning (CS234, 2024, Lecture 1)
- Policy Gradient Foundations: Trajectory Distributions & High-Dimensional State Spaces
Show 22 moreShow fewer
- Generalized Eigenvectors, Diagonalization, and Jordan Canonical Form
- Particle Filtering
- A Look Ahead
- Markov Chains Continued Further (Statistics 110, Lecture 33)
- Bayes' Nets: Sampling
- Hidden Markov Models
- Complexity and Computability with Closed Timelike Curves
- Gibbs Measures on Infinite Trees and Homoclinic Points
- Controlling Stochastic Biophysical Processes, from Protein Folding to Evolution
- Privacy Amplification for Correlated-Noise Mechanisms via b-Min-Sep Subsampling
- Convex Optimization II: Localization and Cutting-Plane Methods
- Ross Recovery Theorem
- Leveraging Per-Instance Privacy for Machine Unlearning
- LTFs and Noise Stability
- Stochastic dynamics
- Fluctuation Relation at the Beat of a Current
- Operational Dosage: The Impact of Capacity Constraints on RCT Evaluation
- Universal Thermodynamic Bounds on Symmetry Breaking in Living Systems
- Introduction to Information Theory
- Frenetic Steering
- Linear Algebra
- Geometric Design Principle of Oscillation-driven Catalytic Molecular Machines