Sign inSign in
ConceptAI & ML

DPO failure cases (first token cancellation)

DPO fails to learn tokens that are identical at the start of both chosen and rejected completions because their respective gradients cancel out in the loss formulation.
Watch it explained

Know first

Named as a prerequisite by the lectures that define this. The number is how many say so. A grey one has no page yet.

Direct preference optimization1Gradient cancellation1Sequence generation1

Lecturers

1

    Connections

    2

    The lectures that define it or assume it, and the concepts next to it.

    1 lecturer. Definitions are written by AI from each lecture’s transcript. A time with a play mark starts the lecture at the moment the lecturer says it.