ConceptAI & ML
Multi-headed attention
Using multiple self-attention layers simultaneously to allow a model to pay attention to different facets of the input data at the same time.
Know first
Named as a prerequisite by the lectures that define this. The number is how many say so. A grey one has no page yet.
Lecturers
2Each in their own terms, from the easiest level.
Ashish Vaswani, Lucas Beyer & William Fedus
Multi-headed attention computes attentions in parallel by splitting representations into groups, allowing the model to learn more than one relation.
Connections
4The lectures that define it or assume it, and the concepts next to it.