Spelled-Out Intro to Neural Networks & Backpropagation: Building micrograd
Founding Member of OpenAI, Former Director of AI at Tesla
2:34:45·2022·undergraduate level
Demystifies PyTorch's autograd in 150 lines of clean Python by manually computing numerical derivatives, applying the multivariable chain rule, and verifying gradients against torch.Tensor.
Step-by-step live derivation and implementation of reverse-mode automatic differentiation over a dynamic expression DAG, culminating in training a multi-layer perceptron from scratch.
Referenced inHacker News
Comments
Add a comment...Sign in