Andrej Karpathy

Founder of Eureka Labs, Former Director of AI at Tesla

Eureka Labs Stanford University Tesla (presenting at CVPR)Independent / Eureka Labs

1 course · 4 standalone talks · 15 total lectures · 27h recorded

AI & Machine Learning Mathematics Engineering, Control & Robotics

Courses by Andrej Karpathy

All courses →

Complete university lecture series taught by Andrej Karpathy.

Neural Networks: Zero to Hero

Independent / Eureka Labs · NN-Z2H11 lectures · 20h

AI & ML·undergraduate

Neural Networks: Zero to Hero

Andrej Karpathy·Independent / Eureka Labs

From scalar autograd and backpropagation in pure Python up to modern Transformer language models built line-by-line from scratch.

Start lecture 1

Standalone talks by Andrej Karpathy

4

Self-contained public lectures, keynote addresses, and research seminars.

CS231n Winter 2016: Lecture 1 - Introduction to Convolutional Neural Networks

AI & ML·Standalone Lecture

CS231n Winter 2016: Lecture 1 - Introduction to Convolutional Neural Networks

Andrej Karpathy·Stanford University

Andrej Karpathy introduces Stanford's CS231n course, providing an overview of convolutional neural networks and their historical context in visual recognition.

Backpropagation and Neural Networks I

AI & ML·Standalone Lecture

Backpropagation and Neural Networks I

Andrej Karpathy·Stanford University

An foundational exposition on the backpropagation algorithm and the initial principles of neural network architecture, delivered as part of Stanford's CS231n course.

Deep Dive into Large Language Models: Pretraining to RLHF

AI & ML·Standalone Lecture

Deep Dive into Large Language Models: Pretraining to RLHF

Andrej Karpathy·Eureka Labs

End-to-end engineering dissection of tokenization, distributed Transformer pretraining, supervised fine-tuning, reinforcement learning on chain-of-thought, and inference failure modes.

Andrej Karpathy on Scalability in Autonomous Driving at CVPR 2020

Engineering·Conference Talk

Andrej Karpathy on Scalability in Autonomous Driving at CVPR 2020

Andrej Karpathy·Tesla (presenting at CVPR)

Andrej Karpathy delivers a keynote at CVPR 2020, detailing the challenges and strategies for achieving scalability in autonomous driving systems.

Study paths featuring Andrej Karpathy

1

All study paths →

Mathematical Foundations of Machine Learning

7 steps · 8h4 universities

EditorialAI & ML·graduate

Mathematical Foundations of Machine Learning

Linear geometry, convex duality, statistical learning, and deep optimization

MIT · Stanford University · Caltech +1

7 curated lecturesOpen path →

All lectures by Andrej Karpathy

15

Search or browse every recorded session across courses and talks.

Showing 15 of 15 lectures

Lecturej/k navigate · c chapters · s save · x watched

Scholar · University

Chapters · Duration

Let's Build GPT: From Scratch, in Code, Spelled Out

Let's Build GPT: From Scratch, in Code, Spelled Out

NN-Z2H · Lec 7 · Andrej Karpathy · Eureka Labs

Andrej Karpathy Eureka Labs

1:56:20

Let's Reproduce GPT-2 (124M): Distributed PyTorch, FlashAttention & FineWeb

Let's Reproduce GPT-2 (124M): Distributed PyTorch, FlashAttention & FineWeb

NN-Z2H · Lec 10 · Andrej Karpathy · Eureka Labs

Andrej Karpathy Eureka Labs

4:01:26

Building makemore Part 3: Activations, Gradients, Kaiming Init & BatchNorm

Building makemore Part 3: Activations, Gradients, Kaiming Init & BatchNorm

NN-Z2H · Lec 4 · Andrej Karpathy · Eureka Labs

Andrej Karpathy Eureka Labs

1:55:57

Spelled-Out Intro to Neural Networks & Backpropagation: Building micrograd

Spelled-Out Intro to Neural Networks & Backpropagation: Building micrograd

NN-Z2H · Lec 1 · Andrej Karpathy · Eureka Labs

Andrej Karpathy Eureka Labs

2:34:45

Building makemore Part 4: Becoming a Backprop Ninja (Manual Tensor Gradients)

Building makemore Part 4: Becoming a Backprop Ninja (Manual Tensor Gradients)

NN-Z2H · Lec 5 · Andrej Karpathy · Eureka Labs

Andrej Karpathy Eureka Labs

1:55:24

Spelled-Out Intro to Language Modeling: Building makemore (Bigram & Cross-Entropy)

Spelled-Out Intro to Language Modeling: Building makemore (Bigram & Cross-Entropy)

NN-Z2H · Lec 2 · Andrej Karpathy · Eureka Labs

Andrej Karpathy Eureka Labs

1:57:45

Let's Build the GPT Tokenizer: Byte Pair Encoding & Unicode Internals

Let's Build the GPT Tokenizer: Byte Pair Encoding & Unicode Internals

NN-Z2H · Lec 9 · Andrej Karpathy · Eureka Labs

Andrej Karpathy Eureka Labs

2:13:34

CS231n Winter 2016: Lecture 1 - Introduction to Convolutional Neural Networks

CS231n Winter 2016: Lecture 1 - Introduction to Convolutional Neural Networks

AI & ML·Standalone Lecture · Andrej Karpathy · Stanford University

Andrej Karpathy Stanford University

1:19:08

Backpropagation and Neural Networks I

Backpropagation and Neural Networks I

AI & ML·Standalone Lecture · Andrej Karpathy · Stanford University

Andrej Karpathy Stanford University

1:19:39

Building makemore Part 2: Multi-Layer Perceptron Language Models (Bengio et al. 2003)

Building makemore Part 2: Multi-Layer Perceptron Language Models (Bengio et al. 2003)

NN-Z2H · Lec 3 · Andrej Karpathy · Eureka Labs

Andrej Karpathy Eureka Labs

1:15:39

Deep Dive into Large Language Models: Pretraining to RLHF

Deep Dive into Large Language Models: Pretraining to RLHF

AI & ML·Standalone Lecture · Andrej Karpathy · Eureka Labs

Andrej Karpathy Eureka Labs

3:31:23

State of GPT: Pretraining, SFT, Reward Modeling & RLHF Pipeline

State of GPT: Pretraining, SFT, Reward Modeling & RLHF Pipeline

NN-Z2H · Lec 8 · Andrej Karpathy · Eureka Labs

Andrej Karpathy Eureka Labs

42:40

Building makemore Part 5: Hierarchical Dilated Causal Convolutions (WaveNet)

Building makemore Part 5: Hierarchical Dilated Causal Convolutions (WaveNet)

NN-Z2H · Lec 6 · Andrej Karpathy · Eureka Labs

Andrej Karpathy Eureka Labs

56:21

Introduction to Large Language Models

Introduction to Large Language Models

NN-Z2H · Lec 11 · Andrej Karpathy · Eureka Labs

Andrej Karpathy Eureka Labs

59:48

Andrej Karpathy on Scalability in Autonomous Driving at CVPR 2020

Andrej Karpathy on Scalability in Autonomous Driving at CVPR 2020

Engineering·Conference Talk · Andrej Karpathy · Tesla (presenting at CVPR)

Andrej Karpathy Tesla (presenting at CVPR)

27:39