“Designing Randomized Trials to Understand Treatment Effect Heterogeneity,” Elizabeth Tipton, Northwestern University

QUANTITATIVE RESEARCH METHODS WORKSHOP

Abstract: Randomized trials are now common in medicine, the social sciences, and in policy related research. The primary goal of a randomized trial is to estimate the average treatment effect, which averages over unit-specific treatment impacts that might be quite heterogenous. Often the results of such a trial are to be used for making policy or practice decisions in a target population; this, paired with the fact that most such trials take place in samples of convenience, has led to an increased interest in methods for generalizing causal effects. This has included new methods for estimation of average treatment effects that take into account population data, as well as methods for designing sampling and recruitment strategies for such studies. But if treatment effects actually vary, often researchers seek to understand if such heterogeneity can be predicted or explained by unit characteristics. In this talk, I focus on this sampling problem in the face of heterogeneity. I show that by methods for optimal designs found in response surface models can be useful, too, in designing sampling plans that result in increased power and precision for these moderator effects. I situate this in an example based on an evaluation of a school-based reading program.

Elizabeth Tipton is an Associate Professor of Statistics, the Co-Director of the Statistics for Evidence-Based Policy and Practice (STEPP) Center, and a Faculty Fellow in the Institute for Policy Research at Northwestern University. She is a Board Member of the Society for Research on Educational Effectiveness (2020 - 2022), is an Associate Editor at the Journal of Educational and Behavioral Statistics, and is on the editorial boards of the Journal of Research on Educational Effectiveness, Research Synthesis Methods, and Psychological Bulletin. She is the recipient of the Frederick Mosteller Award from The Campbell Collaboration, and early career awards from the American Education Research Association, the American Psychological Association, and then Society for Research Synthesis Methods.

This virtual workshop is open to the Yale community. To receive Zoom information, you must subscribe to the Quantitative Research Methods Workshop at this link: https://csap.yale.edu/quantitative-research-methods-workshop.

The series is sponsored by the ISPS Center for the Study of American Politics and The Whitney and Betty MacMillan Center for International and Area Studies at Yale with support from the Edward J. and Dorothy Clarke Kempf Fund.