Sign inSign in
ConceptAI & ML

Transformer

A neural network architecture that uses self-attention mechanisms to process sequences in parallel, often replacing recurrent networks.
as Russ Tedrake defines itMITThe wording closest to all 8 definitions
Watch it explained

Know first

Named as a prerequisite by the lectures that define this. The number is how many say so. A grey one has no page yet.

Opens up

Concepts that name this one as a prerequisite.

Lecturers

8

Each in their own terms, from the easiest level.

  • Andrej Karpathy

    Undergraduate

    The neural network architecture that does all the heavy lifting under the hood for modern language models.
    • 2:29Let's Build GPT: From Scratch, in Code, Spelled Out
  • Geoffrey Hinton

    University of Toronto · Undergraduate

    A kind of neural network architecture introduced by Google researchers in 2017 capable of advanced natural language tasks.
    • 14:30Digital vs. Biological Intelligence (Vienna Gödel Lecture)
  • Ashish Vaswani, Lucas Beyer & William Fedus

    Stanford University · Undergraduate

    A neural network architecture that uses self-attention to model dependencies between tokens in a sequence, allowing for parallel computation.
    • 17:00Transformers United: DL Models that have revolutionized NLP, CV, RL
  • Phillip Isola

    MIT · Undergraduate

    The standard deep learning architecture that relies on attention mechanisms and operates over tokens to capture both local and global relationships.
    • 0:17Architectures: Transformers
  • Gillian Tett

    Santa Fe Institute · Undergraduate

    A type of deep learning architecture based on self-attention mechanisms used for probabilistic prediction systems in modern AI.
    • 1:19:38Whom Do We Trust? How AI Is (Re)Shaping Our Interactions Today
  • Sergey Levine

    UC Berkeley · Graduate

    A sequence model that processes tokens through position-wise encoders, masked self-attention layers, and nonlinear transformations to output token distributions.
    • 0:50RL with Sequence Models & Language Models, Part 2
  • Drew Jaegle

    Stanford University · Graduate

    A general-purpose neural network architecture that uses self-attention mechanisms without relying on task-specific spatial convolutions.
    • 3:07DeepMind's Perceiver and Perceiver IO: New data family architecture

Lectures that assume it

34

The first 12 are shown.

Connections

52

The lectures that define it or assume it, and the concepts next to it.

Concepts10

Show 4 more

Lectures42

Show 36 more

8 lecturers. Definitions are written by AI from each lecture’s transcript. A time with a play mark starts the lecture at the moment the lecturer says it.