Opening Overview of Transformers
Overview of Transformers
- Length
- 1 h 17 min
- Level
- Undergraduate
- Recorded
- Stanford University2024
Chapter 1 of 11: Before the lecture
Walks from query–key–value attention to emergent abilities, chain-of-thought prompting and multi-agent failure modes.
Transformer models replaced recurrent networks with parallelizable self-attention, and this CS25 session at Stanford University surveys that shift and what followed. Steven Feng and fellow presenters Div Garg, Emily Bunnapradist and Seonghee Lee start from rule-based chatbots and word embeddings, then cover queries, keys and values, multi-head attention and cross-attention. Emergent abilities are weighed against the argument that emergence is a mirage, alongside chain-of-thought prompting, mechanistic interpretability and model editing. The second half turns to AI agents: vector-database memory, planning, tool use, and multi-agent communication in natural language. Plan divergence, hallucination and data contamination close the survey, which assumes neural network fundamentals.
Transformer models replaced recurrent networks with parallelizable self-attention, and this CS25 session at Stanford University surveys that shift and what followed. Steven Feng and fellow presenters Div Garg, Emily Bunnapradist and Seonghee Lee start from rule-based chatbots and word embeddings, then cover queries, keys and values, multi-head attention and cross-attention. Emergent abilities are weighed against the argument that emergence is a mirage, alongside chain-of-thought prompting, mechanistic interpretability and model editing. The second half turns to AI agents: vector-database memory, planning, tool use, and multi-agent communication in natural language. Plan divergence, hallucination and data contamination close the survey, which assumes neural network fundamentals.
Attention Is All You Need
Emergent Abilities of Large Language Models
Are Emergent Abilities of Large Language Models a Mirage?

This lecture
Talk 12 of 21 of Transformers United.
Choose a node to see how it connects to this lecture. What comes before is on the leftat the top. What comes after is on the rightat the bottom. The ring is what it teaches. People and places are above. Other lectures and books are below.
Press a chosen node again to open it.
Your comment is kept while you sign up.Sign in
Direct Preference Optimization: Your Language Model is Secretly a Reward Model
Whisper
11 Labs
Textbooks Are All You Need
arXiv:2306.11644
New York Times copyright lawsuit against OpenAI
LLMs Can Self-Improve
https://arxiv.org/abs/2202.05262
https://arxiv.org/abs/2204.14146
https://arxiv.org/abs/2201.11903
https://arxiv.org/abs/2305.10601
https://arxiv.org/abs/2305.14999
Gorilla