Sign inSign in
ConceptAI & ML

Mixture of Experts

A neural network architecture where different subnetworks (experts) specialize in different inputs, coordinated by a gating network or router.
as Albert Jiang defines itStanford UniversityThe wording closest to all 4 definitions
Watch it explained

Know first

Named as a prerequisite by the lectures that define this. The number is how many say so. A grey one has no page yet.

Opens up

Concepts that name this one as a prerequisite.

Lecturers

4

Each in their own terms, from the easiest level.

  • Eric Evans

    Domain-Driven Design Europe · Undergraduate

    An alternative model architecture that routes different inputs to specialized subnetworks.
    • 9:16Opening Keynote: Navigating Change in Domain-Driven Design
  • Steven Feng

    Stanford University · Undergraduate

    A model architecture combining multiple smaller expert neural networks along with a gating network to route inputs to the best suited expert.
    • 23:45Overview of Transformers
  • Ashish Vaswani, Lucas Beyer & William Fedus

    Stanford University · Graduate

    An architecture where a router or gating network computes to which of kk out of nn total experts to send each token.
    • 3:14Mixture of Experts (MoE) paradigm and the Switch Transformer

Lectures that assume it

1

Connections

13

The lectures that define it or assume it, and the concepts next to it.

4 lecturers. Definitions are written by AI from each lecture’s transcript. A time with a play mark starts the lecture at the moment the lecturer says it.