Skip to content
lectern.tv
Simons Institute: Theoretical Foundations of Modern Machine Learning →0/19
1

Talk by Pranjal Awasthi (Google) (SIMONS-TCS-ML)

38:15
2

Talk by Katrina Ligett (Hebrew University of Jerusalem) (SIMONS-TCS-ML)

36:41
3

Efficient Algorithms for Reliable Machine Learning (SIMONS-TCS-ML)

37:26
4

Computational Thinking on Learning Models (SIMONS-TCS-ML)

38:30
5

On the Komlós Conjecture (SIMONS-TCS-ML)

33:41
6

Multi-objective Learning: An Algorithmic Toolbox for Optimal Predictions Anytime Anywhere! (SIMONS-TCS-ML)

37:40
7

Approximately Packing Dijoins Via Nowhere-Zero Flows (SIMONS-TCS-ML)

33:46
8

Lightning Talks! (SIMONS-TCS-ML)

56:01
9

Venturing out into the world with AI (SIMONS-TCS-ML)

44:41
10

Unfamiliar Terrain (SIMONS-TCS-ML)

29:51
11

Astral Space: Convex Analysis at Infinity (SIMONS-TCS-ML)

44:01
12

AI and Safety (SIMONS-TCS-ML)

29:26
13

The Role of TCS in Understanding (AI) Consciousness (SIMONS-TCS-ML)

43:21
14

The Conscious Turing Machine (CTM), a formally defined Theoretical model of Consciousness (SIMONS-TCS-ML)

45:00
15

Theory of Modern AI: Learning Theoretic, Game Theoretic, and Algorithmic Perspectives (SIMONS-TCS-ML)

40:41
16

An Occam's Razor Principle for Transformers? (SIMONS-TCS-ML)

39:55
17

Red Points and Blue Points (SIMONS-TCS-ML)

38:15
18

Latent Variable models and Subset Smoothing (SIMONS-TCS-ML)

29:26
19

Enhanced and Efficient Reasoning in Large Language Models (SIMONS-TCS-ML)

36:16

An Occam's Razor Principle for Transformers? (SIMONS-TCS-ML)

John LangfordSimons Institute (UC Berkeley)

Professor

39:55·2026·research level

John Langford investigates a novel Occam's Razor principle for transformer models, suggesting that simpler learned states lead to vastly improved generalization and extrapolation, challenging traditional PAC theory.

Theory & AlgosAI & MLLanguage Models & NLPStatistical Learning & CausalityTransformersOccam's Razor

John Langford (Microsoft Research)

The Role of TCS in Modern Machine Learning

The Occam's Razor principle is easily expressed in pac learning bounds via noting that hypothesis description length complexity easily appears in these bounds. With the transformer models underlying modern ai agents it appears that there is a different form of occam's razor principle: transformers that learn a simple state generalize and even extrapolate better. The empirical evidence here is stark: a factor of 3(!) reduction in extrapolation error. But what are the principles behind this? Standard PAC theory appears inapplicable.

Previous · Lecture 15

Theory of Modern AI: Learning Theoretic, Game Theoretic, and Algorithmic Perspectives (SIMONS-TCS-ML)

Next · Lecture 17

Red Points and Blue Points (SIMONS-TCS-ML)

Comments

Add a comment...Sign in