ConceptAI & ML
Weight decay
A regularization method that adds the squared norm of the weights to the in-sample error to penalize large weights and prevent overfitting.
Know first
Named as a prerequisite by the lectures that define this. The number is how many say so. A grey one has no page yet.
Regularization2Gradient descent1L2 regularization1Linear model1Loss function1Neural network activation function1Optimization1Optimizer1
Opens up
Concepts that name this one as a prerequisite.
Lecturers
3Each in their own terms, from the easiest level.
Phillip Isola
An implicit or explicit regularizer that shrinks network weights toward zero during optimization.Andrej Karpathy
A regularization technique that adds a small penalty proportional to the weights' squared magnitude to the loss function to prevent overfitting.
Lectures that assume it
1Connections
14The lectures that define it or assume it, and the concepts next to it.