ConceptAI & ML
Memory bound attention
Attention operations are often limited by memory bandwidth rather than compute power because of the constant movement of large intermediate matrices between HBM and SRAM.
Know first
Named as a prerequisite by the lectures that define this. The number is how many say so. A grey one has no page yet.
Lecturers
1Connections
3The lectures that define it or assume it, and the concepts next to it.