“Empirical Assessment of Overlap in Observational Studies With Statistical Guarantees,” Lihua Lei, UC Berkeley

QUANTITATIVE RESEARCH METHODS WORKSHOP

The reliability of causal inference in observational studies crucially relies on the overlap of baseline covariates (a.k.a. positivity or common support) between different treated groups. Without sufficient overlap, the internal validity is hampered unless the causal estimand is redefined. The current empirical assessment of overlap is typically based on estimated propensity scores. This approach is meaningful only when the model specification for propensity scores is correct and it typically has no formal statistical guarantee due to the lack of uncertainty quantification. In this work we treated the overlap condition as a composite null hypothesis (e.g. that propensity scores lie in [η0, 1 − η0] for all units in study population with a given η0) and developed a family of tests with Type-I error control in finite samples without any assumption on the data generating process, if only the observations are independent and identically distributed. The test provides a provably reliable alarming system that helps practitioners detect the lack of overlap if a desirable overlap condition is rejected. Our test exploits the duality between overlap and performance of the optimal classifier that distinguishes treatment groups using baseline covariates [D’Amour et al., 2017], as well as technical tools from information theory and empirical process theory.

Lihua Lei is a fifth year Ph.D. candidate in the Statistics Department at UC Berkeley, advised by Professors Peter Bickel and Michael Jordan. His research interests include high dimensional statistical inference, causal inference, multiple testing, network clustering, stochastic optimization and resampling methods.

The Quantitative Research Method Workshop series is being sponsored by the ISPS Center for the Study of American Politics and The Whitney and Betty MacMillan Center for International and Area Studies at Yale with support from the Edward J. and Dorothy Clarke Kempf Fund. Lunch is served.