ConceptAI & ML
Regularization
Adding a term or constraint to the loss function to prevent a model from overfitting to training data.
Know first
Named as a prerequisite by the lectures that define this. The number is how many say so. A grey one has no page yet.
Opens up
Concepts that name this one as a prerequisite.
Lecturers
6Each in their own terms, from the easiest level.
Yaser Abu-Mostafa
A technique used to constrain model complexity and combat overfitting.Fei-Fei Li, Justin Johnson & Serena Yeung
Pieter Abbeel
Regularization keeps a large hypothesis space but avoids weird parts of that space where overfitting is likely.Stephen Boyd
Regularization balances objectives like tracking error against penalties on input magnitude and variation in optimization problems.Andrej Karpathy
Augmenting the loss function with a small component called regularization loss to incentivize weights to be near zero.Brian Yu
The process of penalizing hypotheses that are more complex in order to favor simpler, more general hypotheses.
Lectures that assume it
18The first 12 are shown.
- Let's Build GPT: From Scratch, in Code, Spelled Out
- Let's Reproduce GPT-2 (124M): Distributed PyTorch, FlashAttention & FineWeb
- Support Vector Machines
- Training Neural Networks I
- Neural Networks
- Radial Basis Functions
- Learning Representations: A Challenge for Learning Theory, COLT 2013
- Deep Learning Software
- Subgradient Methods for Constrained Problems
- Convex Optimization II: Conjugate Gradient and Truncated Newton Methods
- Stochastic Gradient Descent
- Machine Learning: Who to Nudge: Causal vs. Predictive Targeting in a Field Experiment on Student Financial Aid
Connections
40The lectures that define it or assume it, and the concepts next to it.
Concepts9
Show 3 moreShow fewer
Lectures31
- Harvard CS50’s Artificial Intelligence with Python – Full University Course
- The VC Dimension
- Training Neural Networks II
- Overfitting
- Regularization
- Validation
Show 25 moreShow fewer
- Error and Noise
- Applications of Singular Value Decomposition in LDS and Electrical Engineering
- Spelled-Out Intro to Language Modeling: Building makemore (Bigram & Cross-Entropy)
- Loss Functions and Optimization
- Machine Learning: Naive Bayes
- Approximation and Fitting
- Machine Learning: Neural Networks and Decision Trees
- Let's Build GPT: From Scratch, in Code, Spelled Out
- Let's Reproduce GPT-2 (124M): Distributed PyTorch, FlashAttention & FineWeb
- Support Vector Machines
- Training Neural Networks I
- Neural Networks
- Radial Basis Functions
- Learning Representations: A Challenge for Learning Theory, COLT 2013
- Deep Learning Software
- Subgradient Methods for Constrained Problems
- Convex Optimization II: Conjugate Gradient and Truncated Newton Methods
- Stochastic Gradient Descent
- Machine Learning: Who to Nudge: Causal vs. Predictive Targeting in a Field Experiment on Student Financial Aid
- Least-Norm Solutions of Underdetermined Equations
- Efficient Methods and Hardware for Deep Learning
- On the Phase of the de Sitter Density of States
- Visualizing and Understanding
- Regularized Pricing and Risk Models
- Guest Lecture: Aviral Kumar (UC Berkeley CS 285)