The Imperative to Share Complete Replication Files

Scientific knowledge is only as reliable as the empirical analysis on which it is based.

For the majority of published statistical analyses, however, readers have to trust that the scholars correctly implemented the many stages of analysis since replication files are not available. The reality is that it is not uncommon for the key results of scientific research to be non-reproducible or to arise from errors.

The solution to this problem is simple: require scientists to provide complete replication files so that every step of the analysis, from primary data files to final output, can be reproduced. Doing so will make science more reliable, open, cumulative, and accessible.

In a paper that will be part of a special issue in PS: Political Science & Politics on Data Access and Research Transparency, I discuss in more detail the benefits to science of strong norms supporting the provision of complete replication files. In particular, I want to highlight ISPS’s leadership on this issue; ISPS provides a service for ISPS affiliated research in which ithelps produce and publish complete replication files.

Below is the trend in the provision of replication files at The American Journal of Political Science (AJPS). The increase reflects a change in 2010 of the replication policy and editor. The new policy requires accepted publications to post replication files. The editor, Rick Wilson, has also been actively involved ensuring that the policy is carried out.

I expect that if other journals made a similar commitment to transparency we would observe a similar trend in the replication practices of their publications.