William Fedus
Visiting AI & ML Research Seminar · Stanford University
William Fedus is a professor at Stanford University, specializing in machine learning and the development of transformer models and mixture of experts.
- lectures
- 5
- series
- 1
- hours
- 5
Start here
- Transformer Circuits, Induction Heads, In-Context Learningfrom Raschka, Build a Large Language Model (From Scratch)

- Decision Transformer: Reinforcement Learning via Sequence Modelingfrom Raschka, Build a Large Language Model (From Scratch)

- Self Attention and Non-parametric transformers (NPTs)from Raschka, Build a Large Language Model (From Scratch)

Series and courses
Stanford CS25: Transformers United
In these lectures
- Mechanistic interpretability aims to reverse engineer neural network weights into human-understandable computer programs.Transformer Circuits, Induction Heads, In-Context Learning
- Reinforcement learning can be framed as a sequence modeling problem to leverage the scalability and stability of Transformer architectures.Decision Transformer: Reinforcement Learning via Sequence Modeling
- Most supervised deep learning relies on parametric prediction where test-time outputs depend exclusively on learned parameters rather than the training data itself.Self Attention and Non-parametric transformers (NPTs)
Draws on
People cited in the lectures, at the moment of the citation.
- Catherine OlssonAnthropic colleague cited on the title slide.
- Nelson ElhageAnthropic colleague cited on the title slide.
- Chris OlahLecturer and Anthropic colleague cited on the title slide.
- Lukasz KaiserGoogle researcher cited as a colleague during Aidan Gomez's internship.
- Noam ShazeerGoogle researcher mentioned as working on autoregressive models.
- Jakob UszkoreitGoogle researcher mentioned as part of the transformer project.
Playlists
1About
William Fedus is a professor at Stanford University, specializing in machine learning and the development of transformer models and mixture of experts.
Taught with
Same subject, AI & Machine Learning
Connections
15Their places, courses, books and lectures.
Scholars2
Institutions5
Concept1
Lectures5
- Transformer Circuits, Induction Heads, In-Context Learning
- Decision Transformer: Reinforcement Learning via Sequence Modeling
- Self Attention and Non-parametric transformers (NPTs)
- Transformers United: DL Models that have revolutionized NLP, CV, RL
- Mixture of Experts (MoE) paradigm and the Switch Transformer


