Opening Minimizing a Function Step by Step
Minimizing a Function Step by Step
- Length
- 54 min
- Level
- Graduate
- Recorded
- MIT2019
Chapter 1 of 8: Before the lecture
Strang proves that Newton's method squares the error at every step, then weighs it against steepest descent.
Minimizing a function of many variables starts, in session 21 of Strang's MIT 18.065, from the multivariable Taylor series with its gradient and symmetric Hessian. A parallel setup for vector functions defines the Jacobian matrix, and Newton's method then solves a simple system of equations. Strang analyses the convergence rate and shows the error being squared at every step near the solution. Steepest descent follows as the first-order alternative, stepping along the negative gradient with a learning rate, with Levenberg–Marquardt cited as an approximation to Newton. Convex sets, their intersections, and convex functions tested by positive semi-definite Hessians close the session, which assumes multivariable calculus and linear algebra.
Minimizing a function of many variables starts, in session 21 of Strang's MIT 18.065, from the multivariable Taylor series with its gradient and symmetric Hessian. A parallel setup for vector functions defines the Jacobian matrix, and Newton's method then solves a simple system of equations. Strang analyses the convergence rate and shows the error being squared at every step near the solution. Steepest descent follows as the first-order alternative, stepping along the negative gradient with a learning rate, with Levenberg–Marquardt cited as an approximation to Newton. Convex sets, their intersections, and convex functions tested by positive semi-definite Hessians close the session, which assumes multivariable calculus and linear algebra.

This lecture
Lecture 21 of 34 of Matrix Methods in Data Analysis, Signal Processing, and Machine Learning.
Choose a node to see how it connects to this lecture. What comes before is on the leftat the top. What comes after is on the rightat the bottom. The ring is what it teaches. People and places are above. Other lectures and books are below.
Press a chosen node again to open it.
Your comment is kept while you sign up.Sign in