Research LabSan Francisco, CaliforniaEst. 2024eurekalabs.ai

Eureka Labs

Andrej Karpathy's 'Zero to Hero' deep learning and language model implementation curriculum built from scratch.

University Courses

1

Full-semester syllabi

Recorded Lectures

12

1 standalone talk

Archive Runtime

24h

Unabridged recordings

Faculty & Speakers

1

Avg rigor 96/100

Discipline:

University courses from Eureka Labs

All courses →

Complete semester courses and structured lecture series recorded at Eureka Labs.

Neural Networks: Zero to Hero

Independent / Eureka Labs · NN-Z2H11 lectures · 20h

AI & ML·undergraduate

Neural Networks: Zero to Hero

Andrej Karpathy·Independent / Eureka Labs

From scalar autograd and backpropagation in pure Python up to modern Transformer language models built line-by-line from scratch.

Start lecture 1

Talks & colloquia from Eureka Labs

1

Self-contained research seminars, public lectures, and invited colloquia.

Deep Dive into Large Language Models: Pretraining to RLHF

AI & ML·Standalone Lecture

Deep Dive into Large Language Models: Pretraining to RLHF

Andrej Karpathy·Eureka Labs

End-to-end engineering dissection of tokenization, distributed Transformer pretraining, supervised fine-tuning, reinforcement learning on chain-of-thought, and inference failure modes.

Faculty & visiting speakers at Eureka Labs

1

Professors, researchers, and invited lecturers recorded at Eureka Labs.

Andrej Karpathy

Founder of Eureka Labs, Former Director of AI at Tesla

Guided syllabi featuring Eureka Labs

1

All study paths →

Mathematical Foundations of Machine Learning

7 steps · 8h4 universities

EditorialAI & ML·graduate

Mathematical Foundations of Machine Learning

Linear geometry, convex duality, statistical learning, and deep optimization

MIT · Stanford University · Caltech +1

7 curated lecturesOpen path →

All recorded lectures from Eureka Labs

12

Browse, filter, and sort every lecture from Eureka Labs.

Showing 12 of 12 lectures

Lecturej/k navigate · c chapters · s save · x watched

Scholar · University

Chapters · Duration

Let's Build GPT: From Scratch, in Code, Spelled Out

Let's Build GPT: From Scratch, in Code, Spelled Out

NN-Z2H · Lec 7 · Andrej Karpathy · Eureka Labs

Andrej Karpathy Eureka Labs

1:56:20

Let's Reproduce GPT-2 (124M): Distributed PyTorch, FlashAttention & FineWeb

Let's Reproduce GPT-2 (124M): Distributed PyTorch, FlashAttention & FineWeb

NN-Z2H · Lec 10 · Andrej Karpathy · Eureka Labs

Andrej Karpathy Eureka Labs

4:01:26

Building makemore Part 3: Activations, Gradients, Kaiming Init & BatchNorm

Building makemore Part 3: Activations, Gradients, Kaiming Init & BatchNorm

NN-Z2H · Lec 4 · Andrej Karpathy · Eureka Labs

Andrej Karpathy Eureka Labs

1:55:57

Spelled-Out Intro to Neural Networks & Backpropagation: Building micrograd

Spelled-Out Intro to Neural Networks & Backpropagation: Building micrograd

NN-Z2H · Lec 1 · Andrej Karpathy · Eureka Labs

Andrej Karpathy Eureka Labs

2:34:45

Building makemore Part 4: Becoming a Backprop Ninja (Manual Tensor Gradients)

Building makemore Part 4: Becoming a Backprop Ninja (Manual Tensor Gradients)

NN-Z2H · Lec 5 · Andrej Karpathy · Eureka Labs

Andrej Karpathy Eureka Labs

1:55:24

Spelled-Out Intro to Language Modeling: Building makemore (Bigram & Cross-Entropy)

Spelled-Out Intro to Language Modeling: Building makemore (Bigram & Cross-Entropy)

NN-Z2H · Lec 2 · Andrej Karpathy · Eureka Labs

Andrej Karpathy Eureka Labs

1:57:45

Let's Build the GPT Tokenizer: Byte Pair Encoding & Unicode Internals

Let's Build the GPT Tokenizer: Byte Pair Encoding & Unicode Internals

NN-Z2H · Lec 9 · Andrej Karpathy · Eureka Labs

Andrej Karpathy Eureka Labs

2:13:34

Building makemore Part 2: Multi-Layer Perceptron Language Models (Bengio et al. 2003)

Building makemore Part 2: Multi-Layer Perceptron Language Models (Bengio et al. 2003)

NN-Z2H · Lec 3 · Andrej Karpathy · Eureka Labs

Andrej Karpathy Eureka Labs

1:15:39

Deep Dive into Large Language Models: Pretraining to RLHF

Deep Dive into Large Language Models: Pretraining to RLHF

AI & ML·Standalone Lecture · Andrej Karpathy · Eureka Labs

Andrej Karpathy Eureka Labs

3:31:23

State of GPT: Pretraining, SFT, Reward Modeling & RLHF Pipeline

State of GPT: Pretraining, SFT, Reward Modeling & RLHF Pipeline

NN-Z2H · Lec 8 · Andrej Karpathy · Eureka Labs

Andrej Karpathy Eureka Labs

42:40

Building makemore Part 5: Hierarchical Dilated Causal Convolutions (WaveNet)

Building makemore Part 5: Hierarchical Dilated Causal Convolutions (WaveNet)

NN-Z2H · Lec 6 · Andrej Karpathy · Eureka Labs

Andrej Karpathy Eureka Labs

56:21

Introduction to Large Language Models

Introduction to Large Language Models

NN-Z2H · Lec 11 · Andrej Karpathy · Eureka Labs

Andrej Karpathy Eureka Labs

59:48