ConceptAI & ML
Reinforcement learning from human feedback
A technique that trains a reward model directly from human feedback and uses reinforcement learning to optimize an agent's policy.
Know first
Named as a prerequisite by the lectures that define this. The number is how many say so. A grey one has no page yet.
Reinforcement learning7Language model4Alignment1Assistant model1Finetuning1Neural networks1Pre-training1Pre-training phase1
Opens up
Concepts that name this one as a prerequisite.
Lecturers
9Each in their own terms, from the easiest level.
Andrej Karpathy
RLHF is a training stage that optimizes a language model policy using rewards predicted by a reward model trained on human preferences.Emma Brunskill
Reinforcement learning performed using a learned reward signal derived from human feedback.Jason Wei & Hyung Won Chung
An approach that uses a learned reward model as an objective function instead of strict maximum likelihood estimation.Geoffrey Hinton
Training a model using human feedback to avoid wrong outputs or reward it for solving puzzles.Melanie Mitchell
Training a pre-trained transformer model using human ratings of various generated outputs to guide the model toward preferred responses.Chelsea Finn
A post-training technique using reinforcement learning based on human feedback that can sometimes hurt model calibration.Eric Mitchell
A training methodology that uses human preference feedback to guide language model behavior.Nathan Lambert
Specific technical tool for training ML models from human data.
Lectures that assume it
1Connections
19The lectures that define it or assume it, and the concepts next to it.
Concepts7
- Reinforcement learning
- Language model
- Alignment
- Direct preference optimization
- Adversarial examples
- Distribution sharpening in RLHF
Show 1 moreShow fewer
Lectures12
- State of GPT: Pretraining, SFT, Reward Modeling & RLHF Pipeline
- Introduction to Large Language Models
- Deep Dive into Large Language Models: Pretraining to RLHF
- Introduction to Reinforcement Learning (CS234, 2024, Lecture 1)
- Frontiers of Deep Reinforcement Learning
- Intuitions on Language Models and the History of Transformer Architectures