ConceptAI & ML
Mixture of Experts
A neural network architecture where different subnetworks (experts) specialize in different inputs, coordinated by a gating network or router.
Know first
Named as a prerequisite by the lectures that define this. The number is how many say so. A grey one has no page yet.
Opens up
Concepts that name this one as a prerequisite.
Lecturers
4Each in their own terms, from the easiest level.
Eric Evans
An alternative model architecture that routes different inputs to specialized subnetworks.Steven Feng
A model architecture combining multiple smaller expert neural networks along with a gating network to route inputs to the best suited expert.Ashish Vaswani, Lucas Beyer & William Fedus
An architecture where a router or gating network computes to which of out of total experts to send each token.
Lectures that assume it
1Connections
13The lectures that define it or assume it, and the concepts next to it.