Rational Intelligence Seminar Series
The Rational Intelligence Seminar Series (RISS), seeks to advance the understanding of rationality, efficiency and reliability in machine learning systems. These seminars serve as a forum for discussions and quick dissemination of results.
Learning and Incentives in Human–AI Collaboration
Natalie Collina – PhD Student at The University of Pennsylvania, USA
2025-11-19 at 14:30 (CET)
Keywords: Incentive Aware ML, Human–AI Collaboration
Abstract
As AI systems become more capable, a central challenge is designing them to work effectively with humans. I will first consider collaborative prediction, motivated by a doctor consulting an AI that shares the goal of accurate diagnosis. Even when the doctor and AI have only partial and incomparable knowledge, repeated interaction enables richer forms of collaboration: we give distribution-free guarantees that their combined predictions are strictly better than either alone, with regret bounds against benchmarks defined on their joint information. I will then revisit the alignment assumption itself. If an AI is developed by, say, a pharmaceutical company with its own incentives, how can we encourage helpful behavior? A natural scenario is that the doctor has access to multiple models, each from a different provider. Under a mild ‘market alignment’ assumption—that the doctor’s utility lies in the convex hull of the providers’ utilities—we show that in Nash equilibrium of this competition, the doctor can achieve the same outcomes as if a perfectly aligned provider were present. Based on joint work: Tractable Agreement Protocols (STOC’25), Collaborative Prediction (SODA’26), and Emergent Alignment via Competition (in submission).
About the Speaker
I’m a 5th-year PhD student in Computer Science at the University of Pennsylvania, where I am fortunate to be advised by Michael Kearns and Aaron Roth. I work on problems at the intersection of AI and Game Theory. I’m especially interested in understanding repeated strategic interactions between human and AI agents. My work has been awarded with an IBM PhD Fellowship in Trustworthy AI and a joint Best Paper Award/Best Student Paper Award from the ACM Conference for Economics and Computation. I have been recognized as a Rising Star in EECS and my work has been featured in Quanta magazine.