“Agnostic Notes on Regression Adjustments to Experimental Data: Reexamining Freedman’s Critique” with Winston Lin, Columbia

QUANTITATIVE RESEARCH METHODS WORKSHOP

Abstract: This talk will be mostly based on my 2013 Annals of Applied Statistics paper, which reexamines David Freedman’s critique of ordinary least squares regression adjustment in randomized experiments. Random assignment is intended to create comparable treatment and control groups, reducing the need for dubious statistical models. Nevertheless, researchers often use linear regression models to adjust for random treatment-control differences in baseline characteristics. The classic rationale, which assumes the regression model is true, is that adjustment tends to reduce the variance of the estimated treatment effect. In contrast, Freedman used a randomization-based inference framework to argue that under model misspecification, OLS adjustment can lead to increased asymptotic variance, invalid estimates of variance, and small-sample bias. My paper shows that in sufficiently large samples, those problems are either minor or easily fixed. Neglected parallels between regression adjustment in experiments and regression estimators in survey sampling turn out to be very helpful for intuition.

Winston Lin (Ph.D., Statistics, UC Berkeley, 2013) is an adjunct associate research scholar in the Department of Political Science at Columbia University. He used to make regression adjustments (always pre-specified) for a living at Abt Associates and MDRC.

This workshop series is being sponsored by the ISPS Center for the Study of American Politics and The Whitney and Betty MacMillan Center for International and Area Studies at Yale with support from the Edward J. and Dorothy Clarke Kempf Fund.