Opening Alan Edelman on the Julia Language for High-Performance Computing
Alan Edelman on the Julia Language for High-Performance Computing
- Length
- 38 min
- Level
- Graduate
- Recorded
- MIT2019
Chapter 1 of 5: Before the lecture
Derives exact derivatives through the Babylonian square root algorithm implemented with dual numbers in Julia.
Automatic differentiation in the Julia language computes exact derivatives by carrying function values and derivatives together as numerical pairs. Edelman, in session 34 of MIT 18.065, builds a custom dual number type and shows how forward mode applies rules such as the quotient rule line by line. The Babylonian square root iteration, run on these pairs, returns the derivative alongside the root, with no symbolic manipulation or finite differences. He then turns to structured matrices, where a symmetric tridiagonal type stores only its diagonal vectors. Reverse mode closes the session, recasting neural network backpropagation as a lower triangular linear solve.
Automatic differentiation in the Julia language computes exact derivatives by carrying function values and derivatives together as numerical pairs. Edelman, in session 34 of MIT 18.065, builds a custom dual number type and shows how forward mode applies rules such as the quotient rule line by line. The Babylonian square root iteration, run on these pairs, returns the derivative alongside the root, with no symbolic manipulation or finite differences. He then turns to structured matrices, where a symmetric tridiagonal type stores only its diagonal vectors. Reverse mode closes the session, recasting neural network backpropagation as a lower triangular linear solve.
WhySwiftForTensorFlow.md
Autodiff: Calculus from another angle

This lecture
Lecture 34 of 34 of Matrix Methods in Data Analysis, Signal Processing, and Machine Learning.
Choose a node to see how it connects to this lecture. What comes before is on the leftat the top. The ring is what it teaches. People and places are above. Other lectures and books are below.
Press a chosen node again to open it.
Your comment is kept while you sign up.Sign in