Triage Score: A Counterfactual Risk Assessment Instrument (SIMONS-ECON)
39:56LLM-Guided Reinforcement Learning for Mastery Learning: Large-Scale Field Evidence (SIMONS-ECON)
35:06Non-parametric Causal Inference in Dynamic Thresholding Designs (SIMONS-ECON)
30:21Agreement and Alignment for Human-AI Collaborative Decision Making (SIMONS-ECON)
45:21Human Values in Clinical Algorithms, Old and New (SIMONS-ECON)
38:46When Prediction Becomes Policy: What Do Risk Models Actually Learn? (SIMONS-ECON)
33:31Evaluating Probabilistic Decision Making by LLMs: A Decision-Theoretic Framework and Application (SIMONS-ECON)
32:16Algorithmic UDAP (SIMONS-ECON)
32:13Operational Dosage: The Impact of Capacity Constraints on RCT Evaluation (SIMONS-ECON)
35:01The Role of Standardized Tests in Admissions: Predictions vs. Decisions (SIMONS-ECON)
33:37Individual Gaps in Prediction Generalization (SIMONS-ECON)
30:41Beyond Prediction Performance: How Modeling Decisions Shape Fairness Outcomes in Statistical Profiling (SIMONS-ECON)
27:56The Feedback Loop of Statistical Discrimination (SIMONS-ECON)
32:03Decision Theoretic Foundations for Human-AI Collaboration (SIMONS-ECON)
31:01Evaluation Design (SIMONS-ECON)
29:16Recommendation in Equilibrium for the NYC High School Match (SIMONS-ECON)
32:51Automating the US Welfare State: Historical Experiments in Technological Federalism (SIMONS-ECON)
35:41On Machine Learning for Prediction and Prioritization in the Allocation of Scarce Societal Resources (SIMONS-ECON)
35:26Decision Theoretic Foundations for Human-AI Collaboration (SIMONS-ECON)
This session discusses decision-theoretic foundations for human-AI collaboration, presenting measurement frameworks rooted in statistical decision theory and information economics for team performance evaluation.
Jessica Hullman (Northwestern University)
Bridging Prediction and Intervention Problems in Social Systems
Judgments from humans and one or more artificial agents are increasingly combined for decisions, with the expectation that the decisions of the team will outperform those of individual agents. However, simple approaches to providing human decision makers with AI support and evaluating team performance can lead to apparent failures in team performance even when the agents are thought to possess complementary information. I’ll discuss measurement frameworks we’ve developed that apply statistical decision theory and information economics to address questions at the human-agent interface, including how to evaluate when a decision-maker appropriately relies on model predictions, when a human or AI agent could better exploit available contextual information, and how to evaluate (and design) explanations in principled ways.
The Feedback Loop of Statistical Discrimination (SIMONS-ECON)
Next · Lecture 15Evaluation Design (SIMONS-ECON)