Skip to content
lectern.tv
MIT 6.7960: Deep Learning →0/20
1

Lecture 01: Introduction to Deep Learning (6.7960)

1:00:51
2

Lecture 02: How to Train a Neural Net (6.7960)

1:19:34
3

Lecture 03: Approximation Theory (6.7960)

1:22:42
4

Lecture 04: Architectures: Grids (6.7960)

1:23:57
5

Lecture 05: Architectures: Graphs (6.7960)

1:21:14
6

Lecture 06: Generalization Theory (6.7960)

1:20:31
7

Lecture 07: Scaling Rules for Optimization (6.7960)

1:20:56
8

Lecture 08: Architectures: Transformers (6.7960)

1:14:35
9

Lecture 09: Hacker's Guide to Deep Learning (6.7960)

1:15:51
10

Lecture 10: Architectures: Memory (6.7960)

1:13:28
11

Lecture 11: Representation Learning: Reconstruction-Based (6.7960)

1:21:04
12

Lecture 12: Representation Learning: Similarity-Based (6.7960)

1:16:20
13

Lecture 13: Representation Learning: Theory (6.7960)

1:15:21
14

Lecture 14: Generative Models: Basics (6.7960)

1:21:18
15

Lecture 15: Generative Models: Representation Learning Meets Generative Modeling (6.7960)

1:20:40
16

Lecture 16: Generative Models: Conditional Models (6.7960)

1:21:32
17

Lecture 17: Generalization: Out-of-Distribution (OOD) (6.7960)

1:04:41
18

Lecture 18: Transfer Learning: Models (6.7960)

1:25:42
19

Lecture 19: Transfer Learning: Data (6.7960)

1:15:44
20

Lecture 20: Scaling Laws (6.7960)

38:23

Lecture 08: Architectures: Transformers (6.7960)

Phillip IsolaMIT

Professor

1:14:35·2026·graduate level

This lecture introduces transformer architectures, explaining the core concepts of tokens, attention mechanisms, and positional codes. It relates transformers to MLPs, GNNs, and CNNs.

AI & MLLanguage Models & NLPDeep LearningTransformersAttention Mechanism

MIT 6.7960 Deep Learning, Fall 2024

Instructor: Phillip Isola

View the complete course: https://ocw.mit.edu/courses/6-7960-deep-learning-fall-2024/

YouTube Playlist: https://www.youtube.com/playlist?list=PLUl4u3cNGP63URZnh5iqBzDTDYPUTQT-8

Previous · Lecture 7

Lecture 07: Scaling Rules for Optimization (6.7960)

Next · Lecture 9

Lecture 09: Hacker's Guide to Deep Learning (6.7960)

Seminar notes & discussion

Sign in to comment

No public seminar notes on this lecture yet.