ConceptAI & ML
Batch normalization
A technique that explicitly normalizes layer activations to have a zero mean and unit variance over a mini-batch.
Know first
Named as a prerequisite by the lectures that define this. The number is how many say so. A grey one has no page yet.
Opens up
Concepts that name this one as a prerequisite.
Backpropagation through batch normalization1Batch normalization backward pass1Batch normalization calibration1Batch normalization limitations1Batch normalization running stats1Instance normalization1Layer normalization1Running mean and standard deviation1Running statistics in batch normalization1Training vs inference mode1
Lecturers
3Each in their own terms, from the easiest level.
Andrej Karpathy
A neural network layer that normalizes activations across a batch and uses running means and variances tracked via momentum updates during training.Fei-Fei Li, Justin Johnson & Serena Yeung
Ali Rahimi
A technique to normalize layer inputs in a neural network to speed up training and reduce internal covariate shift.
Lectures that assume it
4Connections
19The lectures that define it or assume it, and the concepts next to it.