ConceptAI & ML
RMS norm
Root Mean Square normalization scales inputs by their root mean square values without subtracting the mean, mapping data points onto a hypersphere in high dimensions.
Know first
Named as a prerequisite by the lectures that define this. The number is how many say so. A grey one has no page yet.
Euclidean norm1High dimensional space1Layer normalization1Normalization layers1Root mean square1Vector dimension1
Opens up
Concepts that name this one as a prerequisite.
Lecturers
3Each in their own terms, from the easiest level.
Chris Ré
A simplified version of layer normalization that scales inputs by their root mean square without centering them around a mean.Jeremy Bernstein
A norm for a vector in defined as the Euclidean norm scaled by one over the square root of the dimension.
Connections
7The lectures that define it or assume it, and the concepts next to it.