Sign inSign in
ConceptAI & ML

Pre-layer norm

A layer normalization placement where normalization is applied to the residual path before sub-layers rather than after them.
as Ashish Vaswani defines itStanford University
Watch it explained

Know first

Named as a prerequisite by the lectures that define this. The number is how many say so. A grey one has no page yet.

Lecturers

2

Each in their own terms, from the easiest level.

  • Ming Ding

    Stanford University · Undergraduate

    Pre-LayerNorm places the layer norm before the residual connection in the transformer layer.
    • 14:36From Large Language Models to Large Multimodal Models

Connections

5

The lectures that define it or assume it, and the concepts next to it.

2 lecturers. Definitions are written by AI from each lecture’s transcript. A time with a play mark starts the lecture at the moment the lecturer says it.