Transformers United: DL Models that have revolutionized NLP, CV, RL
This introductory lecture for Stanford's CS25 series details the foundational Transformer architecture, including its attention mechanisms, encoder-decoder structure, and essential components. It contextualizes the model's impact across NLP, computer vision, and reinforcement learning applications.
Since their introduction in 2017, transformers have revolutionized Natural Language Processing (NLP). Now, transformers are finding applications all over Deep Learning, be it computer vision (CV), reinforcement learning (RL), Generative Adversarial Networks (GANs), Speech or even Biology. Among other things, transformers have enabled the creation of powerful language models like GPT-3 and were instrumental in DeepMind's recent AlphaFold2, that tackles protein folding.