ConceptAI & ML
Grouped-query attention
An attention mechanism in between multi-head and multi-query attention that uses fewer key and value heads than query heads.
Know first
Named as a prerequisite by the lectures that define this. The number is how many say so. A grey one has no page yet.
Lecturers
1Connections
3The lectures that define it or assume it, and the concepts next to it.