ConceptAI & ML
DPO failure cases (first token cancellation)
DPO fails to learn tokens that are identical at the start of both chosen and rejected completions because their respective gradients cancel out in the loss formulation.
Know first
Named as a prerequisite by the lectures that define this. The number is how many say so. A grey one has no page yet.
Lecturers
1Connections
2The lectures that define it or assume it, and the concepts next to it.