Performance Engineering of Software Systems

This inaugural lecture for MIT's 6.172 course introduces performance engineering principles through an examination of code optimization, specifically contrasting matrix multiplication implementations in Python, Java, and C.

This lecture presents Bentley's Rules for optimizing software performance, covering techniques such as loop unrolling, short-circuiting, fast paths, and compile-time initialization, illustrated with practical code examples.

This lecture, part of MIT's Performance Engineering course, analyzes various bit manipulation techniques and compiler optimizations, demonstrating how to achieve maximal efficiency for critical code segments.

This lecture delineates the entire software-to-hardware execution pipeline, from high-level source code through compilation to machine code, and its interpretation by the underlying computer architecture, including the instruction set. Professor Leiserson elaborates on the interactions between compilers and processor design.

This lecture details the compilation process from C source code to x86-64 assembly, analyzing the intermediate representation of LLVM IR and its translation to machine instructions. Key mapping principles between high-level language constructs and low-level hardware operations are demonstrated.

This lecture examines modern multicore processor architectures and the necessity of parallel programming for high performance. It details the Cilk runtime system, covering its abstraction of processor cores, synchronization primitives, communication protocols, and provably efficient load balancing mechanisms.

This lecture explores the fundamental concepts of races and parallelism in concurrent systems, detailing how cilkscale can be used for determinacy race detection and discussing various types of schedulers for parallel execution.

This lecture systematically develops methods for analyzing the performance of multithreaded algorithms, specifically addressing divide-and-conquer recurrences in the context of Cilk loops, parallel matrix multiplication, and merge sort.

This lecture dissects the Clang/LLVM compilation pipeline, investigating compiler optimizations, diagnostic reports, and case studies in performance failures. It examines how compilers transform code and their limitations.

This session addresses the crucial aspects of reliably measuring software performance, exploring various timing mechanisms and the common pitfalls leading to inaccurate measurements. The use of simulators for performance evaluation is also covered.

This lecture covers diverse storage allocation techniques, including stacks, fixed-size, and variable-size heaps. It further elucidates garbage collection strategies such as reference counting, mark-and-sweep, and stop-and-copy.

This lecture differentiates between malloc() and mmap(), introduces cactus stacks, and explores parallel allocation strategies including global/local heaps and local ownership. It concludes with an examination of incremental, parallel, and concurrent garbage collection.

This session centers on the Cilk runtime system, emphasizing the 'work-first principle' to optimize serial execution even at the cost of additional computation during steals. Performance models and parallel runtime systems are also highlighted.

This lecture explores cache associativity and the ideal cache model, detailing cache-aware techniques such as tiled matrix multiplication and cache-oblivious strategies like divide-and-conquer matrix multiplication for performance optimization.

This lecture illustrates cache-oblivious algorithms through simulations of heat diffusion and 3-point stencil. It also examines the interaction between caching and parallelism within the Cilk framework.

This lecture examines nondeterministic parallel programming, focusing on atomicity implemented via mutexes, determinacy races, and data races. It also addresses deadlock scenarios, including the classic dining philosophers' problem.

This lecture explores non-lock synchronization methods, utilizing memory models, Peterson's algorithm, and instruction reordering. It covers the compare-and-swap operation, lock-free stacks, and the ABA problem.

This lecture explores domain-specific languages (DSLs), their applicability, scheduling optimizations, and autotuning techniques. GraphIt, Halide, and OpenTuner are discussed as examples of effective DSLs.

This lecture investigates speculative parallelism and its applications in parallel alpha-beta search and jamboree search. It concludes with a comprehensive discussion of high-performance computer chess programs.

This lecture, part of MIT's Performance Engineering course, delves into practical strategies for tuning and optimizing algorithms for the Traveling Salesperson Problem. It covers recursive enumeration, algorithm engineering, and effective application of data structures to achieve high performance.

This lecture explores graph optimizations, both algorithmic and locality-based, addressing challenges such as large, sparse, and irregular real-world graphs. It highlights issues with memory access and the context-specific effectiveness of optimizations.