Sign inSign in
ConceptSystems & CS

KV cache

A caching mechanism for key and value vectors in transformer models used to speed up inference.
Watch it explained

Know first

Named as a prerequisite by the lectures that define this. The number is how many say so. A grey one has no page yet.

LLMs2GPU memory1Transformer1

Opens up

Concepts that name this one as a prerequisite.

Lecturers

2

Each in their own terms, from the easiest level.

  • Yonggon Park & Rakesh Nadig

    ETH Zürich · Graduate

    A cache used in transformer models to store key and value tensors for accelerating inference.
    • 40:34Cutting-Edge Research in Flash Memory

Connections

6

The lectures that define it or assume it, and the concepts next to it.

2 lecturers. Definitions are written by AI from each lecture’s transcript. A time with a play mark starts the lecture at the moment the lecturer says it.