Stanford CS25: V1 I Transformers United: DL Models that have revolutionized NLP, CV, RL
22:43Stanford CS25: V1 I Transformers in Vision: Tackling problems in Computer Vision
1:08:36Stanford CS25: V1 I Decision Transformer: Reinforcement Learning via Sequence Modeling
1:20:42Stanford CS25: V1 I Mixture of Experts (MoE) paradigm and the Switch Transformer
1:05:44Stanford CS25: V1 I DeepMind's Perceiver and Perceiver IO: new data family architecture
58:58Stanford CS25: V1 I Self Attention and Non-parametric transformers (NPTs)
1:05:43Stanford CS25: V1 I Transformer Circuits, Induction Heads, In-Context Learning
59:34Neuroscience-Inspired Artificial Intelligence (CS25)
1:22:14How I Learned to Stop Worrying and Love the Transformer (CS25)
1:20:38Neuroscience-Inspired Artificial Intelligence (CS25)
This lecture explores how attention mechanisms within Transformers can approximate sparse distributed memory, drawing parallels to neuroscience for advanced AI architectures. It analyzes the computational mechanisms enabling such approximations.
Attention Approximates Sparse Distributed Memory
Trenton Bricken
PhD student at Harvard
Stanford CS25: V1 I Transformer Circuits, Induction Heads, In-Context Learning
Next · Lecture 9How I Learned to Stop Worrying and Love the Transformer (CS25)
Seminar notes & discussion
No public seminar notes on this lecture yet.