An Observation on Generalization and Unsupervised Compression
Co-Founder of SSI & Former Chief Scientist of OpenAI
57:21·2023·graduate level·Research Seminar
Connects Kolmogorov complexity and Solomonoff induction to next-token prediction, arguing that optimal sequence compression forces models to recover shared world structure.
AI & MLStatistical Learning & CausalityLanguage Models & NLPSimons Institutetheoretical computer science
Ilya Sutskever (OpenAI)
Large Language Models and Transformers
Seminar notes & discussion
No public seminar notes on this lecture yet.