Transformers United: DL Models that have revolutionized NLP, CV, RL
22:43Transformers in Vision: Tackling problems in Computer Vision
1:08:36Decision Transformer: Reinforcement Learning via Sequence Modeling
1:20:42Mixture of Experts (MoE) paradigm and the Switch Transformer
1:05:44DeepMind's Perceiver and Perceiver IO: New data family architecture
58:58Self Attention and Non-parametric transformers (NPTs)
1:05:43Transformer Circuits, Induction Heads, In-Context Learning
59:34Neuroscience-Inspired Artificial Intelligence
1:22:14How I Learned to Stop Worrying and Love the Transformer
1:20:38How I Learned to Stop Worrying and Love the Transformer
Ashish Vaswani, a co-author of "Attention Is All You Need," discusses the foundational motivations and historical evolution of the Transformer model. The session concludes with an overview of promising future research directions in Transformer architectures.
Ashish will present the motivations behind the Transformer and how it's evolved over the years. He will conclude with a few useful research directions.