ConceptAI & ML
Neural network training
The process of organizing many things, labeling them into categories, and finding matching images to fill up those categories.
Know first
Named as a prerequisite by the lectures that define this. The number is how many say so. A grey one has no page yet.
Opens up
Concepts that name this one as a prerequisite.
Lecturers
3Each in their own terms, from the easiest level.
Andrej Karpathy
The computational process of modeling statistical relationships in token sequences by taking windows of tokens and updating weights to predict the next token.Pieter Abbeel
Training a deep neural network is optimized like logistic regression by maximizing log probability using gradient ascent and early stopping on hold-out data.
Lectures that assume it
15The first 12 are shown.
- Building makemore Part 3: Activations, Gradients, Kaiming Init & BatchNorm
- Speculations on Test-Time Scaling for Large Language Models (o1)
- Building makemore Part 2: Multi-Layer Perceptron Language Models (Bengio et al. 2003)
- Let's Reproduce GPT-2 (124M): Distributed PyTorch, FlashAttention & FineWeb
- Training Neural Networks I
- Recurrent Neural Networks
- Training Sand to Think: Artificial General Intelligence & Future of Physics
- How to Train a Neural Net
- Marin: An Open Lab For Frontier AI
- ImageNet and Convolutional Neural Networks
- Generalization: Out-of-Distribution (OOD)
- Efficient Methods and Hardware for Deep Learning
Connections
28The lectures that define it or assume it, and the concepts next to it.
Concepts10
Lectures18
- Deep Dive into Large Language Models: Pretraining to RLHF
- Archetypes and Outcasts in the Work of August Sander
- Machine Learning: Neural Networks and Decision Trees
- Building makemore Part 3: Activations, Gradients, Kaiming Init & BatchNorm
- Speculations on Test-Time Scaling for Large Language Models (o1)
- Building makemore Part 2: Multi-Layer Perceptron Language Models (Bengio et al. 2003)
Show 12 moreShow fewer
- Let's Reproduce GPT-2 (124M): Distributed PyTorch, FlashAttention & FineWeb
- Training Neural Networks I
- Recurrent Neural Networks
- Training Sand to Think: Artificial General Intelligence & Future of Physics
- How to Train a Neural Net
- Marin: An Open Lab For Frontier AI
- ImageNet and Convolutional Neural Networks
- Generalization: Out-of-Distribution (OOD)
- Efficient Methods and Hardware for Deep Learning
- Representation Learning: Theory
- Transformers in Vision: Tackling problems in Computer Vision
- Emergent Behaviors in Foundational Models