Sign inSign in
ConceptAI & ML

Multi-headed attention

Using multiple self-attention layers simultaneously to allow a model to pay attention to different facets of the input data at the same time.
as Brian Yu defines itHarvard University
Watch it explained

Know first

Named as a prerequisite by the lectures that define this. The number is how many say so. A grey one has no page yet.

Lecturers

2

Each in their own terms, from the easiest level.

  • Ashish Vaswani, Lucas Beyer & William Fedus

    Stanford University · Graduate

    Multi-headed attention computes kk attentions in parallel by splitting representations into groups, allowing the model to learn more than one relation.
    • 2:19Self Attention and Non-parametric transformers (NPTs)

Connections

4

The lectures that define it or assume it, and the concepts next to it.

Concepts2

2 lecturers. Definitions are written by AI from each lecture’s transcript. A time with a play mark starts the lecture at the moment the lecturer says it.