ConceptAI & ML
DPO gradient mechanics
The gradient of the DPO loss weighs preference pairs by whether the reward model is currently correct or incorrect, applying higher weights to incorrect predictions.
Know first
Named as a prerequisite by the lectures that define this. The number is how many say so. A grey one has no page yet.
Lecturers
1Connections
4The lectures that define it or assume it, and the concepts next to it.