Opening Distributed Data Warehouse OLAP Databases
Distributed Data Warehouse OLAP Databases
- Length
- 1 h 23 min
- Level
- Undergraduate
- Recorded
- Carnegie Mellon University2023
Chapter 1 of 9: Before the lecture
Pavlo contrasts broadcast and shuffle joins, then cloud warehouses that split object storage from compute.
Distributed OLAP databases keep analytical workloads apart from transactional systems and load data through ETL or ELT pipelines built with tools such as Informatica, dbt, Airbyte and Fivetran. Pavlo opens with Zynga's acquired game studios and Walmart's hurricane-preparation analysis, then models warehouse data as fact and dimension tables in star and snowflake schemas. He contrasts pushing a query to the data with pulling data to the query node, and adds network transfer cost to query planning. Distributed joins follow: replicated tables, co-partitioned data, broadcast joins and shuffle joins. Session 23 of CMU 15-445/645 ends on cloud designs with shared-disk object stores, serverless compute, S3 Select, and decoupled catalogs and formats like Parquet and Arrow.
Distributed OLAP databases keep analytical workloads apart from transactional systems and load data through ETL or ELT pipelines built with tools such as Informatica, dbt, Airbyte and Fivetran. Pavlo opens with Zynga's acquired game studios and Walmart's hurricane-preparation analysis, then models warehouse data as fact and dimension tables in star and snowflake schemas. He contrasts pushing a query to the data with pulling data to the query node, and adds network transfer cost to query planning. Distributed joins follow: replicated tables, co-partitioned data, broadcast joins and shuffle joins. Session 23 of CMU 15-445/645 ends on cloud designs with shared-disk object stores, serverless compute, S3 Select, and decoupled catalogs and formats like Parquet and Arrow.
Farmville / Zynga
Walmart Hurricane Preparation
Amazon S3 Select Documentation
This lecture
Lecture 23 of 24 of Intro to Database Systems.
Choose a node to see how it connects to this lecture. What comes before is on the leftat the top. What comes after is on the rightat the bottom. The ring is what it teaches. People and places are above. Other lectures and books are below.
Press a chosen node again to open it.
Your comment is kept while you sign up.Sign in
MapReduce: Simplified Data Processing on Large Clusters