Sign inSign in
ConceptAI & ML

Tensor cores

A hardware unit and instruction in Volta GPUs that performs 4x4x4 mixed-precision matrix multiply and accumulate operations in a single clock cycle.
as Song Han defines itStanford UniversityThe wording closest to all 3 definitions
Watch it explained

Know first

Named as a prerequisite by the lectures that define this. The number is how many say so. A grey one has no page yet.

GPU architecture2Matrix multiplication2Floating-point operations1Mixed precision training1SIMD execution1

Opens up

Concepts that name this one as a prerequisite.

Lecturers

3

Each in their own terms, from the easiest level.

  • Onur Mutlu

    ETH Zürich · Undergraduate

    Specialized floating-point multiply and accumulate units in modern GPUs designed to accelerate machine learning workloads.
    • 38:18GPU Architectures II & Memory Overview and Technology (ETH Zürich, Digital Design and Computer Architecture, Lecture 20)
  • Andrej Karpathy

    Graduate

    Specialized hardware instructions in the A100 architecture that perform 4x4 matrix multiplications efficiently.
    • 1:29:10Let's Reproduce GPT-2 (124M): Distributed PyTorch, FlashAttention & FineWeb

Connections

7

The lectures that define it or assume it, and the concepts next to it.

3 lecturers. Definitions are written by AI from each lecture’s transcript. A time with a play mark starts the lecture at the moment the lecturer says it.