Transformers & High-Dimensional Vector Geometry of LLMs
Geometric dissection of token embeddings, unembedding matrices, high-dimensional almost-orthogonality (Johnson-Lindenstrauss intuition), and Transformer residual streams.
Breaking down how Large Language Models work, visualizing how data flows through.
Instead of sponsored ad reads, these lessons are funded directly by viewers: https://3b1b.co/support
---
Seminar notes & discussion
No public seminar notes on this lecture yet.