Opening Language Models: Prompting, Chain-of-Thought, and Instruction-Tuning
Language Models: Prompting, Chain-of-Thought, and Instruction-Tuning
- Length
- 1 h 17 min
- Level
- Undergraduate
- Recorded
- MIT2026
Chapter 1 of 10: Before the lecture
Andreas weighs whether in-context learning retrieves learned tasks or runs a learning algorithm inside the transformer.
Language models, from n-gram counts to large generative transformers, are the subject of this guest lecture by Jacob Andreas in MIT's 6.7960 Deep Learning course. Andreas starts with next-word prediction and count-based models such as Brants et al.'s 2007 machine translation work. He then covers masked language modeling in BERT and the scaling of GPT-3, Gopher and MT-NLG. The middle section treats few-shot prompting, the debate over what in-context learning actually computes, and chain-of-thought prompting, where the phrase "let's think step by step" improves arithmetic accuracy. The last part covers instruction-tuning and reinforcement learning from human feedback, with a reward model trained on preferences over summaries. Basic deep learning, neural network architectures and probability are assumed.
Language models, from n-gram counts to large generative transformers, are the subject of this guest lecture by Jacob Andreas in MIT's 6.7960 Deep Learning course. Andreas starts with next-word prediction and count-based models such as Brants et al.'s 2007 machine translation work. He then covers masked language modeling in BERT and the scaling of GPT-3, Gopher and MT-NLG. The middle section treats few-shot prompting, the debate over what in-context learning actually computes, and chain-of-thought prompting, where the phrase "let's think step by step" improves arithmetic accuracy. The last part covers instruction-tuning and reinforcement learning from human feedback, with a reward model trained on preferences over summaries. Basic deep learning, neural network architectures and probability are assumed.
Large Language Models in Machine Translation
LaMDA
GPT-3
Jurassic
Gopher
MT-NLG 530B
BERT paper
This lecture
Lecture 21 of 24 of Deep Learning.
Choose a node to see how it connects to this lecture. What comes before is on the leftat the top. What comes after is on the rightat the bottom. The ring is what it teaches. People and places are above.
Press a chosen node again to open it.
Your comment is kept while you sign up.Sign in
ELMo paper
Emergent Abilities of Large Language Models
Language Models are Few-Shot Learners
Calibrate Before You Use: Improving Few-Shot Performance of Language Models
Zhao et al. '21
Min et al., 2022
Zhang et al., 2022
Lu et al., 2022
Brown et al. '20
Wei et al. '22
Chain-of-Thought Prompting Elicits Reasoning in Large Language Models
Large Language Models are Zero-Shot Reasoners
InstructGPT
Kojima et al., '22
TruthfulQA
Simple statistical gradient-following algorithms for connectionist reinforcement learning