Sign inSign in
ConceptAI & ML

Model distillation

Transferring knowledge from a large complex teacher model to a smaller, more efficient student model.
as Steven Feng defines itStanford University
Watch it explained

Know first

Named as a prerequisite by the lectures that define this. The number is how many say so. A grey one has no page yet.

LLMs1Neural networks1Pretraining data synthesis1

Opens up

Concepts that name this one as a prerequisite.

Lecturers

2

Each in their own terms, from the easiest level.

  • Song Han

    Stanford University · Undergraduate

    Using multiple large teacher neural networks to teach a small student neural network so it can perform as well as the larger models.
    • 50:47Efficient Methods and Hardware for Deep Learning

Connections

4

The lectures that define it or assume it, and the concepts next to it.

Concepts2

Lectures2

2 lecturers. Definitions are written by AI from each lecture’s transcript. A time with a play mark starts the lecture at the moment the lecturer says it.