“Agreement and Alignment for Human-AI Collaboration,” Aaron Roth, UPenn

QUANTITATIVE RESEARCH METHODS WORKSHOP

Abstract: As AI models become increasingly powerful, it is an attractive proposition to use them in important decision making pipelines, in collaboration with human decision makers. But how should a human being and a machine learning model collaborate to reach decisions that are better than either of them could achieve on their own? If the human and the AI model were perfect Bayesians, operating in a setting with a commonly known and correctly specified prior, Aumann’s classical agreement theorem would give us one answer: they could engage in conversation about the task at hand, and their conversation would be guaranteed to converge to (accuracy-improving) agreement. This classical result however would require making many implausible assumptions, both about the knowledge and computational power of both parties. We show how to recover similar (and more general) results using only computationally and statistically tractable assumptions, which substantially relax full Bayesian rationality. In the second part of the talk, we go on to consider a more difficult problem: that the AI model might be acting at least in part to advance the interests of its designer, rather than the interests of its user, which might sometimes be in tension. We show how market competition between different AI providers can mitigate this problem assuming only a mild “market alignment” assumption — that the user’s utility function lies in the convex hull of the AI providers utility functions — even when no single provider is well aligned. In particular, we show that in all Nash equilibria of the AI providers under this market alignment condition, the user is able to advance her own goals as well as she could have in collaboration with a perfectly aligned AI model. This talk describes the results of three papers, which are joint works with Natalie Collina, Ira Globus-Harris, Surbhi Goel, Varun Gupta, Emily Ryu, and Mirah Shi. Link to related papers: PAPER 1 | PAPER 2 | PAPER 3

Aaron Roth is the Henry Salvatori Professor of Computer and Cognitive Science, in the Computer and Information Sciences department at the University of Pennsylvania, with a secondary appointment in the Wharton statistics department. He is affiliated with the Warren Center for Network and Data Science, and co-director of the Networked and Social Systems Engineering (NETS) program.  He is also an Amazon Scholar at Amazon AWS. He is the recipient of the Hans Sigrist Prize, a Presidential Early Career Award for Scientists and Engineers (PECASE), an Alfred P. Sloan Research Fellowship, an NSF CAREER award, and research awards from Yahoo, Amazon, and Google.  His research focuses on the algorithmic foundations of data privacy, algorithmic fairness, game theory, learning theory, and machine learning.  Together with Cynthia Dwork, he is the author of the book “The Algorithmic Foundations of Differential Privacy.” Together with Michael Kearns, he is the author of “The Ethical Algorithm.”

This workshop is being sponsored jointly with the Center for Algorithms, Data, and Market Design at Yale (CADMY).

The Quantitative Research Methods Workshop is open to the Yale community. To receive announcements and invitations to attend, please subscribe at this link.

The Quantitative Research Methods Workshop series is sponsored by the ISPS Center for the Study of American Politics and The Whitney and Betty MacMillan Center for International and Area Studies at Yale with support from the Edward J. and Dorothy Clarke Kempf Fund.