/
lectern.tv
Play15s30s1x0:00/--:--
DoneWatched
Expand
lectern.tv
0:18/1:19:35
11:30
14:50
11:50
8:20
33:05
Introduction & MapReduce
Taught by RTM himself at MIT: dry, surgical systems thinking on partial failures, idempotency, network partitions, and the trade-offs between consistency and performance.
Introduces the core concepts of distributed systems and analyzes Google's 2004 MapReduce paper. It covers the motivations for building distributed systems—parallelism, fault tolerance, physical distribution, and security isolation—and examines key design challenges including concurrency, partial failures, and achieving scalable speedup. Robert Morris also details the mechanics of the MapReduce framework, including workers, the master process, the Google File System (GFS), and the shuffle phase.
Lecture Guide
- 1Distributed systems are built primarily for parallelism, fault tolerance, physical separation, and security isolation, but should be avoided if a single computer can solve the problem.
- 2Scalability aims for proportional performance increases with added resources, though bottlenecks like database limits or network throughput often constrain scaling.
- 3Fault tolerance in large-scale systems requires handling constant component failures through non-volatile storage and replication.
- 4Consistency models range from strong consistency (guaranteeing reads see the most recent write) to weak consistency, balancing correctness against the high communication costs of distributed replicas.
Proofs, Examples & Demos