Transformers United: DL Models that have revolutionized NLP, CV, RL
22:43Transformers in Vision: Tackling problems in Computer Vision
1:08:36Decision Transformer: Reinforcement Learning via Sequence Modeling
1:20:42Mixture of Experts (MoE) paradigm and the Switch Transformer
1:05:44DeepMind's Perceiver and Perceiver IO: New data family architecture
58:58Self Attention and Non-parametric transformers (NPTs)
1:05:43Transformer Circuits, Induction Heads, In-Context Learning
59:34Neuroscience-Inspired Artificial Intelligence
1:22:14How I Learned to Stop Worrying and Love the Transformer
1:20:38Self Attention and Non-parametric transformers (NPTs)
This session from Stanford's CS25 course introduces the foundational intuitions behind Transformer architectures, then delves into the theoretical and practical aspects of Non-Parametric Transformers (NPTs), a model recently accepted at NeurIPS.
Aidan will chat about the Transformer origins and intuitions for 15 minutes. Then Neil and Jannik will take over to talk about Non-Parametric Transformers (NPTs). NPTs have just been accepted to NeurIPs and you can check them out at
DeepMind's Perceiver and Perceiver IO: New data family architecture
Next · Lecture 7Transformer Circuits, Induction Heads, In-Context Learning