ConceptAI & ML
Supervised fine-tuning
Training stage where the model learns how to behave in a certain way using examples of the harness, tools, and ideal responses.
Know first
Named as a prerequisite by the lectures that define this. The number is how many say so. A grey one has no page yet.
Opens up
Concepts that name this one as a prerequisite.
Direct preference optimization2Reinforcement learning2Continual learning catastrophic forgetting1Discriminator - generator gap1Online fine-tuning1Reasoning models1Reinforcement learning (RL) in LLMs1Reinforcement learning (RL) in math1Reinforcement learning (RL) training1Reinforcement learning from human feedback1Reward model1Reward modeling1
Lecturers
4Each in their own terms, from the easiest level.
Andrej Karpathy
Supervised fine-tuning is the stage where a base model is trained on a high-quality dataset of prompt-response pairs written by contractors to learn assistant behavior.Tristan Buckmaster
A training process where models imitate curated responses including reasoning traces written by other models.Nathan Lambert
Training a model to learn task-specific capabilities, usually via autoregressive LM loss.
Lectures that assume it
6- How Can Robots Learn to Interact and Coordinate with Others?
- POPRi: Private Federated Learning using Preference-Optimized Synthetic Data
- Frontiers of Deep Reinforcement Learning
- Reinforcement Learning from Human Feedback: Algorithms & Applications
- Inference Methods for Deep Learning
- Beyond LLMs: Agents, Emergent Abilities, Intermediate-Guided Reasoning, BabyLM
Connections
23The lectures that define it or assume it, and the concepts next to it.
Concepts10
Lectures13
- Let's Build GPT: From Scratch, in Code, Spelled Out
- State of GPT: Pretraining, SFT, Reward Modeling & RLHF Pipeline
- Let's Reproduce GPT-2 (124M): Distributed PyTorch, FlashAttention & FineWeb
- Deep Dive into Large Language Models: Pretraining to RLHF
- A 'Deep Blue-Kasparov Moment for Mathematics': Euler Equations and AI
- Aligning Open Language Models
Show 7 moreShow fewer
- How Always-On Agents Work
- How Can Robots Learn to Interact and Coordinate with Others?
- POPRi: Private Federated Learning using Preference-Optimized Synthetic Data
- Frontiers of Deep Reinforcement Learning
- Reinforcement Learning from Human Feedback: Algorithms & Applications
- Inference Methods for Deep Learning
- Beyond LLMs: Agents, Emergent Abilities, Intermediate-Guided Reasoning, BabyLM