ConceptAI & ML
Reward modeling
The process of reconstructing scalar reward values from large datasets of human choices to reflect human preferences.
Know first
Named as a prerequisite by the lectures that define this. The number is how many say so. A grey one has no page yet.
Opens up
Concepts that name this one as a prerequisite.
Lecturers
2Each in their own terms, from the easiest level.
Andrej Karpathy
Reward modeling involves training a model to predict a scalar reward representing the quality of a completion given a prompt, based on human preference rankings.
Lectures that assume it
4Connections
14The lectures that define it or assume it, and the concepts next to it.
Concepts8
Lectures6
- State of GPT: Pretraining, SFT, Reward Modeling & RLHF Pipeline
- What Humans Want: Models of Preference and Reward in Minds and Machines
- Speculations on Test-Time Scaling for Large Language Models (o1)
- Deep Dive into Large Language Models: Pretraining to RLHF
- AI Seminar: Actionable Interpretability for Language Model Reasoning
- Overview of Transformers (Stanford CS25: Transformers United V6)