Vanishing Gradient
Defined in 2 lectures. Each definition is in that lecture's terms.
Definitions are written by AI from each lecture’s transcript. The time opens the lecture where the lecturer explains it.
- A scenario where gradients become extremely small, causing optimization progress to be very slow and dominated by noise.
- A problem where weight matrices are very small, causing gradients to shrink exponentially close to zero during backpropagation and preventing stable training.