/
lectern.tv
Play15s30s1x0:00/--:--
DoneWatched
Expand
lectern.tv
29:09/1:19:34
1:26
3:08
3:21
2:00
3:15
3:44
2:05
1:37
5:32
2:58
1:01
0:41
2:10
3:13
1:48
0:51
1:15
0:45
2:03
0:36
4:31
0:54
4:41
1:55
0:45
2:42
3:36
0:37
1:35
1:40
2:25
1:43
4:07
3:44
How to Train a Neural Net
Learn how neural network training actually works under the hood through backpropagation, computational graphs, and differentiable programming.
Explains the mechanics of training neural networks, covering gradient descent, stochastic gradient descent (SGD), momentum, and loss landscapes. It then introduces computational graphs and details how backpropagation operates through chains, multi-layer perceptrons, and directed acyclic graphs (DAGs) using matrix calculus. Finally, it explores differentiable programming, parameter sharing, and optimizing inputs versus parameters.
Referenced inMIT OpenCourseWare · MIT 6.7960 (Fall 2024)
Lecture Guide
- 1Gradient descent finds parameters by moving iteratively in the direction of maximal slope on a loss landscape.
- 2Stochastic gradient descent approximates full dataset gradients using small random subsets or batches, trading stability for speed.
- 3Backpropagation uses the chain rule and dynamic programming to efficiently compute and propagate gradients across shared terms in a computation graph.
- 4Training a linear layer involves multiplying forward by weights and backward by transposed weights.
Proofs, Examples & Demos
Part of course·Lecture 2 of 20
6.7960: Deep Learning
About the lecturer
Professor
Sara Beery is an MIT Professor whose research in deep learning focuses on generalization, out-of-distribution detection, and novel neural network architectures.
1 course · 20 lectures in archiveView profile
Discussion
Share a seminar note, question, or timestamped reference (e.g. 14:20)...Sign in