Opening Shaping the Future of AI from the History of Transformer Architectures
Shaping the Future of AI from the History of Transformer Architectures
- Length
- 37 min
- Level
- Undergraduate
- Recorded
- Stanford University2024
Chapter 1 of 5: Before the lecture
Analyzes the evolution of transformer architectures through compute scaling to explain why simpler decoder models win.
Hyung Won Chung of OpenAI traces how cheaper compute shaped Transformer architectures in Stanford CS25, an undergraduate research seminar. In session 19, Chung compares encoder-decoder and decoder-only models and explains why separate parameters can become unnecessary at scale.
Hyung Won Chung of OpenAI traces how cheaper compute shaped Transformer architectures in Stanford CS25, an undergraduate research seminar. In session 19, Chung compares encoder-decoder and decoder-only models and explains why separate parameters can become unnecessary at scale.
Scaling Instruction-Finetuned Language Models

This lecture
Talk 19 of 21 of Transformers United.
Choose a node to see how it connects to this lecture. What comes before is on the leftat the top. What comes after is on the rightat the bottom. The ring is what it teaches. People and places are above. Other lectures and books are below.
Press a chosen node again to open it.
Your comment is kept while you sign up.Sign in