ConceptAI & ML
Neural network layer
A layer computes a linear transformation of the input vector via a weight matrix and bias vector, followed by a non-linear activation function.
Know first
Named as a prerequisite by the lectures that define this. The number is how many say so. A grey one has no page yet.
Opens up
Concepts that name this one as a prerequisite.
Lecturers
3Each in their own terms, from the easiest level.
Andrej Karpathy
A fundamental building block of a neural network that performs matrix multiplication of inputs by weights, followed by adding a bias and applying a non-linearity.Chris Ré
A neural network layer is a transformation of input data using weights and biases followed by an activation function.
Lectures that assume it
12- Building makemore Part 3: Activations, Gradients, Kaiming Init & BatchNorm
- Let's Build GPT: From Scratch, in Code, Spelled Out
- CNN Architectures
- Training Neural Networks I
- Recurrent Neural Networks, Transformers, and Attention
- Demystifying Mixtral of Experts
- An Observation on Generalization and Unsupervised Compression
- Approximation Theory
- Scaling Rules for Optimization
- Introduction to Deep Learning
- Aligning Open Language Models
- Metrized Deep Learning
Connections
24The lectures that define it or assume it, and the concepts next to it.
Concepts9
- Matrix multiplication
- Dot product
- Linear algebra
- Vector
- Activation distribution visualization
- Activation function
Show 3 moreShow fewer
Lectures15
- Spelled-Out Intro to Language Modeling: Building makemore (Bigram & Cross-Entropy)
- Neural Networks I: Feedforward Architectures & Backpropagation
- Structure of Neural Networks for Deep Learning
- Building makemore Part 3: Activations, Gradients, Kaiming Init & BatchNorm
- Let's Build GPT: From Scratch, in Code, Spelled Out
- CNN Architectures
Show 9 moreShow fewer
- Training Neural Networks I
- Recurrent Neural Networks, Transformers, and Attention
- Demystifying Mixtral of Experts
- An Observation on Generalization and Unsupervised Compression
- Approximation Theory
- Scaling Rules for Optimization
- Introduction to Deep Learning
- Aligning Open Language Models
- Metrized Deep Learning