Mathematical Foundations of Machine Learning
Linear geometry, convex duality, statistical learning, and deep optimization
A cross-institution sequence tracing how column-space geometry (MIT), convex optimization and KKT duality (Stanford), statistical generalization bounds (Caltech), and activation/gradient dynamics form one coherent mathematical edifice.
Curriculum sequence
7
The Geometry of Linear Equations
Establishes the column-space geometry underlying every linear representation.
MIT 18.06 Lecture 5: Transposes, Permutations & Vector Spaces R^n
Develops vector spaces and subspaces needed for orthogonal projections.
Stanford EE364a Lecture 3: Convex Functions — First/Second-Order Conditions & Epigraphs
Connects first- and second-order curvature conditions to global convexity.
Stanford EE364a Lecture 6: Saddle-Point Interpretation, KKT Conditions & Perturbation
Introduces saddle-point geometry, KKT optimality, and Lagrangian duality.
Lecture 01 - The Learning Problem
Formalizes inductive learning from finite samples and hypothesis spaces.
Lecture 02 - Is Learning Feasible?
Uses Hoeffding bounds to explain why out-of-sample generalization is possible.
Building makemore Part 3: Activations, Gradients, Kaiming Init & BatchNorm
Applies variance preservation and gradient flow to deep neural architectures.