Convex Optimization I

Legendary graduate engineering teaching: Boyd blends razor-sharp mathematical precision with street-smart engineering intuition on recognizing hidden convex structure.

Builds geometric fluency over polyhedra, norm balls, ellipsoids, positive semidefinite cones S₊ⁿ, and operations that preserve convexity.

Covers Jensen's inequality, Hessian positive semidefiniteness ∇²f(x) ⪰ 0, log-sum-exp, matrix-fractional functions, and pointwise supremum rules.

Taxonomizes standard problem classes (Linear, Quadratic, Second-Order Cone, and Semidefinite Programs) and equivalent problem transformations.

Why the Lagrange dual problem is ALWAYS convex even when the primal is nonconvex—plus weak duality bounds, Slater's constraint qualification, and strong duality.

Derives Karush-Kuhn-Tucker optimality conditions, complementary slackness, and shadow-price sensitivity analysis ∂p*/∂uᵢ = -λᵢ*.

Professor Stephen Boyd, of the Stanford University Electrical Engineering department, expands upon his previous lectures on convex optimization problems for the course, Convex Optimization I (EE 364A).

This lecture, part of Stanford's EE364A series, introduces the fundamental concepts of duality in convex optimization, exploring its theoretical underpinnings and practical applications within electrical engineering.

This lecture delves into the theoretical foundations and practical applications of duality theory within convex optimization, examining optimality conditions and alternative theorems. Students explore the relationship between primal and dual problems to achieve robust solutions.

The session explores methods for approximation and fitting data using convex optimization, including least-squares, linear, and quadratic programs. Focus is placed on formulating these problems to achieve optimal curve fitting and model estimation.

This lecture, part of Stanford's EE 364A, elucidates the application of statistical estimation techniques within the framework of convex optimization, detailing problem formulations and solution methodologies.

This lecture introduces the application of convex optimization to geometric problems in engineering, including topics such as extremal volume and minimax problems. Students learn to formulate and solve spatial optimization challenges effectively.

This lecture, part of Stanford's EE 364A, delves into advanced geometric problems within the framework of convex optimization. It systematically applies principles of convex analysis to formulate and solve engineering design challenges, including specific applications relevant to signal processing and control systems.

This lecture provides essential background in numerical linear algebra, laying the groundwork for understanding the computational aspects of convex optimization. Key concepts like matrix factorizations and iterative solvers are reviewed.

This session explores methods for unconstrained minimization in convex optimization, covering algorithms like gradient descent and Newton's method. The focus is on finding optimal solutions without explicit boundary conditions.

This lecture focuses on minimization problems with equality constraints, introducing Lagrange multipliers and KKT conditions for finding optimal solutions. Practical applications in electrical engineering are also explored.

Continuing from the previous lecture, this session delves deeper into advanced techniques and complex scenarios involving equality-constrained minimization. It examines algorithmic refinements and specific engineering applications.

This lecture introduces interior-point methods, a powerful class of algorithms for solving convex optimization problems with inequality constraints. The barrier method and primal-dual algorithms are covered in detail.

The final lecture of EE364a reviews key concepts and extends to advanced applications of convex optimization in signal processing, control, and circuit design. It consolidates theoretical understanding with practical implementation challenges.

This introductory lecture for EE364B covers course logistics and then immediately dives into the crucial concept of subgradients for non-differentiable convex functions. Students learn their definition and basic properties.

This session continues the exploration of subgradients, examining their calculus rules, optimality conditions, and applications in extending gradient-based methods. It provides a deeper understanding of handling non-smoothness.

This lecture introduces the fundamental subgradient methods for minimizing non-differentiable convex functions, including convergence analysis and practical implementation challenges. Emphasis is placed on their iterative nature.