Gradient Clipping
Defined in 2 lectures. Each definition is in that lecture's terms.
Definitions are written by AI from each lecture’s transcript. The time opens the lecture where the lecturer explains it.
- A mechanism that scales down gradients exceeding a maximum magnitude to prevent unstable updates and overshooting.
- A mitigation technique for exploding gradients that scales back very large gradients to constrain them.