“Data Quality in Online Surveys: The Case of Survey Trolls,” Sunshine Hillygus, Duke University
QUANTITATIVE RESEARCH METHODS WORKSHOP
Abstract: Online surveys now represent a large segment of the academic research on public attitudes and behaviors. There is widespread recognition that online surveys can be plagued by lower levels of data quality than traditional survey modes, especially low levels of respondent attention that can introduce noise into survey estimates. We evaluate a variety of metrics of data quality and introduce another source of measurement error: survey trolling, or mischievous responding, whereby respondents give insincere answers in an effort to be provocative or humorous. We offer a technique for identifying mischievous responding and evaluate the implications for survey research. Our analysis examines the incidence and implications of survey trolling across multiple surveys, with a particular focus on evaluating the extent to which it biases estimates of political misinformation in the American public.
D. Sunshine Hillygus is Professor of Political Science and Public Policy and director of the Initiative on Survey Methodology at Duke University. Prof. Hillygus specializes in public opinion, political communication, political behavior, and survey methodology. She is co-author of Making Young Voters: Converting Civic Attitudes into Civic Actions (forthcoming, Cambridge University Press), The Persuadable Voter: Wedge Issues in Political Campaigns (Princeton University Press, 2008), and The Hard Count: The Social and Political Challenges of the 2000 Census (Russell Sage Foundation, 2006). Her work has also been published in the American Journal of Political Science, Public Opinion Quarterly, Political Analysis, Statistical Science, Annals of Applied Statistics, Journal of Survey Statistics and Methodology, among others. She is associate PI of the 2020 American National Election Study, associate editor of Political Analysis, chair of the Advisory Committee for Public Opinion Quarterly, and formerly served on the Scientific Advisory Committee for the U.S. Census Bureau. She received her Ph.D. in political science from Stanford University in 2003.
This workshop series is being sponsored by the ISPS Center for the Study of American Politics and The Whitney and Betty MacMillan Center for International and Area Studies at Yale with support from the Edward J. and Dorothy Clarke Kempf Fund.
The workshop is open to Yale faculty, students, and professional staff only.