/
lectern.tv
Play15s30s1x0:00/--:--
DoneWatched
Expand
lectern.tv
1:52:20/2:34:45
14:20
17:40
20:00
35:00
1:07:45
Spelled-Out Intro to Neural Networks & Backpropagation: Building micrograd
Demystifies PyTorch's autograd in 150 lines of clean Python by manually computing numerical derivatives, applying the multivariable chain rule, and verifying gradients against torch.Tensor.
Provides a step-by-step live derivation and implementation of reverse-mode automatic differentiation over a dynamic expression DAG, culminating in building a scalar-valued autograd engine called micrograd and training a multi-layer perceptron from scratch in pure Python.
Lecture Guide
- 1Automatic differentiation evaluates gradients by recursively applying the calculus chain rule backward through a dynamic computation graph.
- 2Mathematical expressions, neurons, layers, and multi-layer perceptrons can be represented using simple recursive data structures in Python.
- 3During backpropagation, gradient contributions from multiple paths must be accumulated via addition rather than overwritten.
- 4Gradient descent iteratively updates parameters by taking small steps in the direction opposite to the loss gradient to minimize error.
Proofs, Examples & Demos