ConceptAI & ML
Monte Carlo methods
Learning methods that update value estimates by waiting until the end of the episode and using the actual total return.
Know first
Named as a prerequisite by the lectures that define this. The number is how many say so. A grey one has no page yet.
Opens up
Concepts that name this one as a prerequisite.
Bias/variance trade-off1Bootstrapping1Cesàro's method for pi1First-Visit Monte-Carlo policy evaluation1Least squares Monte-Carlo1Monte Carlo expected improvement approximation1Monte Carlo policy evaluation1Monte Carlo subgradient method1Monte-Carlo roll-Outs1N-step Sarsa1Noisy unbiased subgradient1Sarsa()1
Lecturers
2Each in their own terms, from the easiest level.
Emily Belli
Computations by statistical sampling.
Lectures that assume it
9- Model-Free Control — ε-Greedy SARSA & Off-Policy Q-Learning
- Speculations on Test-Time Scaling for Large Language Models (o1)
- Assignment, State, and Side-effects (SICP Lecture 5a)
- Value Function Approximation & Deep Q-Networks (DQN)
- Natural Policy Gradients, Covariant Updates & The Fisher Information Matrix
- Convex Optimization II: Stochastic Programming and Cutting-Plane Methods
- Subgradients: Subdifferential Calculus & Optimality Conditions
- Surrogate Modeling and Bayesian Optimization (Part 2)
- Classic Games
Connections
19The lectures that define it or assume it, and the concepts next to it.
Concepts8
- Markov decision processes
- Probability density
- Value function
- Bias/variance trade-off
- Bootstrapping
- Cesàro's method for pi
Show 2 moreShow fewer
Lectures11
- Model-Free Prediction — Monte Carlo, TD(0) & TD(λ)
- Dwarfs of Computation in HPC Fusion Codes
- Model-Free Control — ε-Greedy SARSA & Off-Policy Q-Learning
- Speculations on Test-Time Scaling for Large Language Models (o1)
- Assignment, State, and Side-effects (SICP Lecture 5a)
- Value Function Approximation & Deep Q-Networks (DQN)