ConceptAI & ML
Linear layer
A computational building block that computes a weighted sum of inputs plus a bias term.
Know first
Named as a prerequisite by the lectures that define this. The number is how many say so. A grey one has no page yet.
Opens up
Concepts that name this one as a prerequisite.
Lecturers
2Each in their own terms, from the easiest level.
Andrej Karpathy
A linear layer performs a transformation on the incoming data.
Lectures that assume it
6- Let's Build GPT: From Scratch, in Code, Spelled Out
- Let's Build the GPT Tokenizer: Byte Pair Encoding & Unicode Internals
- Building makemore Part 5: Hierarchical Dilated Causal Convolutions (WaveNet)
- Let's Reproduce GPT-2 (124M): Distributed PyTorch, FlashAttention & FineWeb
- How to Train a Neural Net
- Overview of Transformers
Connections
16The lectures that define it or assume it, and the concepts next to it.
Concepts8
Show 2 moreShow fewer
Lectures8
- Building makemore Part 3: Activations, Gradients, Kaiming Init & BatchNorm
- Introduction to Deep Learning
- Let's Build GPT: From Scratch, in Code, Spelled Out
- Let's Build the GPT Tokenizer: Byte Pair Encoding & Unicode Internals
- Building makemore Part 5: Hierarchical Dilated Causal Convolutions (WaveNet)
- Let's Reproduce GPT-2 (124M): Distributed PyTorch, FlashAttention & FineWeb