AI Seminar: Actionable Interpretability for Language Model Reasoning

Language models are increasingly capable of complex reasoning, yet their successes and failures offer only a partial picture of what they learn and how to improve them. How can a deeper understanding of model behavior guide the development of more capable and trustworthy systems? In this talk, I will present my research on actionable interpretability: turning an understanding of how models reason, learn, and fail into targeted improvements in training and inference. I will briefly survey this approach across evaluation, post-training, and inference, then focus on three post-training advances. I will show how localizing task-relevant computation enables parameter-efficient adaptation, how analyzing training dynamics reveals why distillation should broaden a model's solution coverage before reinforcement learning sharpens it, and how gradient-based analysis exposes reward hacking and guides its suppression. These results illustrate how interpretability can inform not only our understanding of language models, but also practical methods for making their reasoning more efficient, effective, and reliable.

BIO: Xi Ye is an Assistant Professor in the Department of Computing Science at the University of Alberta. He is also an AMII fellow, where he also holds a Canada CIFAR AI Chair. Before joining the University of Alberta in 2026, he spent two years as a postdoctoral fellow at Princeton Language and Intelligence. He received his Ph.D. in Computer Science from the University of Texas at Austin in 2024. His research focuses on language models, reasoning, and interpretability. He has served as a Senior Area Chair for EMNLP and as an Area Chair for ICLR, ACL, EMNLP, and NAACL. He also co-organized the Natural Language Reasoning and Structured Explanations workshop at ACL 2024 and gave a tutorial on Explanation in the Era of Large Language Models at NAACL 2024.