ConceptAI & ML
PyTorch tensors
Multi-dimensional arrays provided by the PyTorch library that allow efficient manipulation and GPU acceleration.
Know first
Named as a prerequisite by the lectures that define this. The number is how many say so. A grey one has no page yet.
Opens up
Concepts that name this one as a prerequisite.
Bigram language model1Character-level language model (Bi-gram)1CUDA model transfer1Custom autograd functions in PyTorch1Floating-point precision in deep learning1Indexing backward pass1Matrix multiplication broadcasting1Normalization1PyTorch `torch.nn`1PyTorch variables1Torch.tril1View backward pass1
Lecturers
3Each in their own terms, from the easiest level.
Fei-Fei Li, Justin Johnson & Serena Yeung
Imperative ndarrays that run on the GPU without built-in notions of computational graphs, gradients, or deep learning.Jamie Meindl
PyTorch Tensors store multidimensional arrays of numbers, similar to NumPy arrays, but with support for GPU acceleration and automatic differentiation.
Lectures that assume it
5- Spelled-Out Intro to Neural Networks & Backpropagation: Building micrograd
- Let's Build GPT: From Scratch, in Code, Spelled Out
- Building makemore Part 4: Becoming a Backprop Ninja (Manual Tensor Gradients)
- Building makemore Part 5: Hierarchical Dilated Causal Convolutions (WaveNet)
- Let's Reproduce GPT-2 (124M): Distributed PyTorch, FlashAttention & FineWeb
Connections
17The lectures that define it or assume it, and the concepts next to it.
Concepts9
Lectures8
- Spelled-Out Intro to Language Modeling: Building makemore (Bigram & Cross-Entropy)
- Deep Learning Software
- PyTorch Tutorial
- Spelled-Out Intro to Neural Networks & Backpropagation: Building micrograd
- Let's Build GPT: From Scratch, in Code, Spelled Out
- Building makemore Part 4: Becoming a Backprop Ninja (Manual Tensor Gradients)