“Empirical Bayes for Adaptive Experiments,” Karun Adusumilli, UPenn

QUANTITATIVE RESEARCH METHODS WORKSHOP

Abstract: We investigate Empirical Bayes analysis in the context of compound adaptive experiments, where the arm distribution in each experiment follows a normal distribution with an unknown mean parameter that we aim to estimate. Our results establish that g-modeling, even when incorrectly assuming exogenous data collection, produces a valid Empirical Bayes procedure, irrespective of the sampling algorithm employed or the endogeneity of sample sizes. In contrast, the f-modeling approach results in biased estimates due to the adaptivity of the sampling process. Remarkably, we show that the risk guarantees established for independent and identically distributed (iid) data remain valid for adaptively generated data. Our procedure requires no prior knowledge of the sampling algorithm used to generate the data, which may even vary across different experiments. We validate the robustness of the g-modeling approach through simulations involving commonly used adaptive algorithms and illustrate its applicability using a real-world dataset comprising multiple sequential experiments.

Karun Adusumilli is an Assistant Professor of Economics at the University of Pennsylvania. His research interests are in Econometrics and Applied Econometrics. In recent years, his areas of interest have included analysis of adaptive experiments, Reinforcement Learning and models with unobserved heterogeneity.

This workshop is open to the Yale community. To receive announcements and invitations to attend, please subscribe at https://csap.yale.edu/quantitative-research-methods-workshop.

The Quantitative Research Methods Workshop series is sponsored by the ISPS Center for the Study of American Politics and The Whitney and Betty MacMillan Center for International and Area Studies at Yale with support from the Edward J. and Dorothy Clarke Kempf Fund.