QUANTITATIVE RESEARCH METHODS WORKSHOP: Jake Bowers (U Illinois), “Machine Learning and Causal Inference: A Modular Approach to Improving Precision in Experiments”

“Machine Learning and Causal Inference: A Modular Approach to Improving Precision in Experiments”

Jake Bowers, Associate Professor in the Department of Political Science and the Department of Statistics, University of Illinois at Urbana-Champaign, and a 2015-16 Fellow on the White House Social and Behavioral Sciences Team

Abstract: The design of a randomized study guarantees not only clear and “interpretable comparisons”(Kinder and Palfrey, 1993, page 7) but valid statistical tests even in the absence of large samples or known data generating processes for outcomes (Fisher, 1935, Chap 2). Yet, while design alone yields valid tests the tests could lack power: a valid but wide confidence interval may be more useful than a misleadingly narrow confidence interval, but still shed little light on the theory motivating the study. After a brief demonstration of Fisher’s statistical framework, we show a method by which a researcher may use substantive background knowledge about outcomes in order to increase the power of her statistical tests. Combining substance and design in this particular way enables valid and powerful tests. We combine modern methods of machine learning with Fisher’s conceptual framework and survey sampling based design-based statistical inference originating with Neyman in order to maximize power without compromising the integrity of the resulting statistical inference. We apply our ideas in the context of a natural experiment created by the London subway bombings of 2005.

This workshop series is being sponsored by the ISPS Center for the Study of American Politics and The Whitney and Betty MacMillan Center for International and Area Studies at Yale with support from the Edward J. and Dorothy Clarke Kempf Fund.