Query Optimization in Modern Database Systems
Prof. Andy PavloAdvanced systems course on relational query optimization, analyzing System R bottom-up dynamic programming, Volcano/Cascades top-down rule engines, cardinality estimation, and join ordering.
Syllabus
14Lecture 1: SQL Optimization Background (15-799)
This lecture establishes fundamental concepts of relational database query optimization, including algebraic transformations and the role of cost-based optimizers in enhancing query performance.
Lecture 2: History of Query Optimizers ft. IBM System R (15-799)
This session explores the historical evolution of query optimizers, focusing on the seminal contributions and architectural design of IBM System R and its influence on modern database systems.
Lecture 3: IBM Starburst Query Rewriter + Optimizer (15-799)
This lecture delves into the IBM Starburst project, examining its innovative query rewriting techniques and extensible optimizer framework, which advanced declarative query processing.
Lecture 4: EXODUS + Volcano Query Optimizer Generators (15-799)
This lecture analyzes the EXODUS and Volcano optimizer generators, highlighting their rule-based query rewrite engines and the iterator model for efficient query plan execution.
Lecture 5: Cascades Query Optimizer (15-799)
This session presents the Cascades framework for query optimization, detailing its dynamic programming approach, memo structure, and use of logical and physical properties for plan enumeration.
Lecture 6: Query Plan Transformations (15-799)
This lecture examines various query plan transformations, including predicate pushdown, join reordering, and subquery decorrelation, crucial for optimizing the logical query execution path.
Lecture 7: Join Ordering: Bottom-up (15-799)
This lecture introduces bottom-up strategies for join ordering, including the use of dynamic programming and bushy tree enumeration to find optimal join sequences for multi-way joins.
Lecture 8: Join Ordering: Top-Down / Transformational (15-799)
This session explores top-down and transformational approaches to join ordering, focusing on heuristic search methods and rule-based systems to determine efficient execution plans.
Lecture 9: Search Parallelization: Bottom-up (15-799)
This lecture discusses bottom-up strategies for parallelizing query plan search, focusing on distributing computation across multiple processors to accelerate optimization.
Lecture 10: Search Parallelization: Top-Down (15-799)
This session covers top-down approaches to parallelizing query plan search, including techniques for breaking down the optimization problem into concurrently executable subproblems.
Lecture 11: Unnesting Queries (15-799)
This lecture focuses on techniques for unnesting subqueries, converting complex nested SQL statements into equivalent, more efficient join-based queries for better performance.
Lecture 12: Query Cost Models: Statistics (15-799)
This session explores the fundamental role of statistical information in query cost models, discussing how data distributions and metadata inform optimizer decisions.
Lecture 13: Query Cost Models: Cardinality Estimation (15-799)
This lecture delves into cardinality estimation techniques for query cost models, analyzing methods like histograms and sampling to predict result set sizes accurately.
Lecture 14: Query Cost Models: More Cardinality Estimation (15-799)
This session continues the discussion on advanced cardinality estimation, covering techniques like correlation analysis and learned models to refine cost predictions in complex queries.













