ConceptAI & ML
Reward model
A separate neural network trained to imitate human preferences and scores for model rollouts.
Know first
Named as a prerequisite by the lectures that define this. The number is how many say so. A grey one has no page yet.
Opens up
Concepts that name this one as a prerequisite.
Lecturers
5Each in their own terms, from the easiest level.
Matt Jones
A model built up from human rankings of possible outputs to nudge post-training mathematics toward more agreeable responses.Andrej Karpathy
Emma Brunskill
A function that defines the scalar feedback signal received by an agent for taking a specific action in a specific state.Jacob Andreas
A model trained to reproduce human annotators' judgments about which outputs are good and which are bad.Sergey Levine
A neural network function used to score any possible completion or answer for a given prompt in an open-vocabulary setting.
Lectures that assume it
1Connections
17The lectures that define it or assume it, and the concepts next to it.
Concepts10
Lectures7
- Let's Build GPT: From Scratch, in Code, Spelled Out
- RL with Sequence Models & Language Models, Part 2
- Deep Dive into Large Language Models: Pretraining to RLHF
- AI as Therapist? Intimacy Without Understanding
- Introduction to Reinforcement Learning (CS234, 2024, Lecture 1)
- Language Models: Prompting, Chain-of-Thought, and Instruction-Tuning