“Measuring Polarization in High-Dimensional Data: Method and Application to Congressional Speech,” Jesse M. Shapiro, Brown University

BEHAVIORAL SCIENCES WORKSHOP

We study trends in the partisanship of congressional speech from 1873 to 2016. We define partisanship to be the ease with which an observer could infer a congressperson’s party from a fixed amount of speech, and we estimate it using a structural choice model and methods from machine learning. Our method corrects a severe finite-sample bias that we show arises with standard estimators. The results reveal that partisanship is far greater in recent years than in the past, and that it increased sharply in the early 1990s after remaining low and relatively constant over the preceding century. Our method is applicable to the study of high-dimensional choices in many domains, and we illustrate its broader utility with an application to residential segregation.

Jesse Shapiro is the George S. and Nancy B. Parker Professor of Economics at Brown University. Prior to joining Brown University in 2015 he was the Chookaszian Family Professor of Economics at the University of Chicago Booth School of Business. Shapiro received his BA in economics in 2001 and his PhD in economics in 2005 from Harvard University. He is a Research Associate at the National Bureau of Economic Research and a former editor of the Journal of Political Economy. He was a 2011-12 Alfred P. Sloan Research Fellow.

The workshop is held jointly between the Yale departments of Economics, Political Science, Psychology, and the School of Management (SOM). The workshop is cosponsored by the Center for the Study of American Politics (CSAP) and the School of Management’s International Center for Finance and Whitebox Advisors fund. Lunch will be served.