/
lectern.tv
Play15s30s1x0:00/--:--
DoneWatched
Expand
lectern.tv
13:08/25:23
1:50
3:16
4:54
2:45
9:07
2:23
1:08
Applications of Optimization in Machine Learning and Engineering
A high-level survey from the University of Washington on the diverse applications of optimization theory across modern machine learning, physics, and engineering disciplines.
Lecture guide · Written by AI from the lecture. For the lecture’s own words, open the Transcript tab.
Provides a high-level overview of optimization theory and its applications across machine learning, control theory, and inverse problems. Steve Brunton surveys how optimization forms the engine behind training models, encoding physical laws, stabilizing dynamical systems, solving seismic and medical imaging problems, and developing digital twins for engineering design.
This lecture assumes
- Optimizationtaught in Optimization: A Bootcamp for Machine Learning, Inverse Problems, and Control at 1:04
- Neural Networkstaught in Artificial Intelligence at 43:38
- Loss Functiontaught in Spelled-Out Intro to Neural Networks & Backpropagation: Building micrograd at 1:52:20
- Lagrange Multiplierstaught in Optimization: A Bootcamp for Machine Learning, Inverse Problems, and Control at 19:18
- Machine learningtaught in Just Machine Learning at 2:45 and 1 more
Key Takeaways
- Machine learning training can be formulated as an optimization problem that minimizes the mismatch between model predictions and actual data.
- Physical laws and domain knowledge can be incorporated into machine learning models either by adding penalty terms to the loss function or through explicit constraint equations.
- Control theory frames the design of actuators and controllers as a constrained optimization problem satisfying system dynamics.
- Inverse problems involve using observed data and a forward model to back out unknown parameters, and can be solved effectively using optimization techniques.
Concepts Explained11
Machine Learning Training as Optimization—Training a machine learning model is formulated as an optimization problem where parameters theta are tuned to minimize the mismatch between model predictions and actual output data.
Least Squares Regression—A historical precursor to machine learning where unknown coefficients x are found by minimizing the sum of squared errors over data.
Physics-Informed Machine Learning—Encoding known physics, such as partial differential equations, into a neural network by adding governing physical residual terms to the loss function.
Constrained Optimization for Physical Models—Setting up explicit constraint equations that must be satisfied for a reduced-order model to be physical, solved using techniques like KKT conditions.
Examples, Proofs & Derivations
Works & Sources Cited9
Paper
Physics Informed Machine Learning—Raissi, Perdikaris, Karniadakis, JCP 378, 2023
Paper
Constrained sparse Galerkin regression—Loiseau & SLB, JFM 838, 2018
Paper
Flexible Muscle-Based Locomotion for Bipedal Creatures—Thomas Geijtenbeek, Michiel van de Panne, Frank van der Stappen, ACM Transactions on Graphics, 2013
Paper
Robust Principal Component Analysis?—Candes et al., J. ACM, 2011
Paper
Paper by Schert et al.—Schert et al., Phys Rev Fluids, 2020
Part of course·Lecture 3 of 9
ME 565: Optimization Bootcamp: Convex Optimization for ML & Control
About the lecturer
Professor of Mechanical Engineering & Adjunct Professor of Applied Mathematics, University of Washington
University of Washington Professor known for data-driven dynamical systems, SINDy, Koopman operator theory, and control engineering.
2 courses · 22 lectures in archiveView profile
Discussion
Share a seminar note, question, or timestamped reference (e.g. 14:20)...Sign in