ConceptAI & ML
Data parallelism
Concurrency that arises from performing the same operation on different pieces of data.
Know first
Named as a prerequisite by the lectures that define this. The number is how many say so. A grey one has no page yet.
Opens up
Concepts that name this one as a prerequisite.
Lecturers
3Each in their own terms, from the easiest level.
Ashish Vaswani, Lucas Beyer & William Fedus
The standard setup in deep learning where the same model weights are replicated across all cores and data is partitioned.Simon Peyton Jones
Operating simultaneously on bulk data with a single flow of control and implicit synchronization.
Lectures that assume it
1Connections
13The lectures that define it or assume it, and the concepts next to it.