Opening Questions for Theory in the New Age of Machine Learning
Questions for Theory in the New Age of Machine Learning
- Length
- 42 min
- Level
- Graduate
- Recorded
- Simons Institute (UC Berkeley)2026
Chapter 1 of 6: Before the lecture
Machine learning theory long assumed that learning means tuning parameters and that weak prior knowledge forces reliance on big data and statistics. Tom Mitchell of Carnegie Mellon argues that learning agents using large language models as subroutines unsettle both assumptions. His first architecture has GPT-4 generate justifications for labels and distil them into a rubric, tested on CK12 data about student hints. A second proposes predictor features, drawing on sources such as Google Trends and FRED, from a target stated in natural language. A third writes code and reflects on its own log files to set new tasks, before Mitchell poses questions for theory built on Valiant's PAC framework.

Your comment is kept while you sign up.Sign in
Mitchell builds three LLM-based learners, among them a classifier that distils GPT-4 justifications into a rubric.
Machine learning theory long assumed that learning means tuning parameters and that weak prior knowledge forces reliance on big data and statistics. Tom Mitchell of Carnegie Mellon argues that learning agents using large language models as subroutines unsettle both assumptions. His first architecture has GPT-4 generate justifications for labels and distil them into a rubric, tested on CK12 data about student hints. A second proposes predictor features, drawing on sources such as Google Trends and FRED, from a target stated in natural language. A third writes code and reflects on its own log files to set new tasks, before Mitchell poses questions for theory built on Valiant's PAC framework.
15-681 Machine Learning Fall 1994 Course Syllabus
Valiant, 1984
This lecture
Talk 21 of 21 of Theoretical Foundations of Modern Machine Learning.
Choose a node to see how it connects to this lecture. What comes before is on the leftat the top. The ring is what it teaches. People and places are above. Other lectures and books are below.
Press a chosen node again to open it.