Building Trust in Science: Yale Hosts Conversations on Replication, Transparency, and Public Confidence
A little more than a decade ago, two cancer‑research labs spent a year unable to reproduce each other’s experiments. Until the two lead researchers flew across the country and discovered that the only difference in their procedures was how vigorously and for how long the labs stirred one enzyme solution.
The incident provided additional insight into a growing, ongoing crisis in which science must contend with threats to its soundness and a skeptical public.
“Trust cannot really be demanded,” said Stuart Buck of The Good Science Project. “Trust has to be earned.”
Last month, the Democratic Innovations program at ISPS partnered with The Data-Intensive Social Science Center, the Tobin Center for Economic Policy, and Yale Library for a pair of events with Buck about how reproducibility, transparency, and humility serve not as bureaucratic burdens in science but as the architecture of trustworthy knowledge.
Barbara Rockenbach, the Stephen F. Gates ’68 University Librarian at Yale, framed the conversations as a necessary corrective to meet this globally significant moment, in which the pace of discovery continues to accelerate, and the stakes of successful navigation have never been higher.
“Across disciplines, we’re seeing growing concerns about scientific integrity, from falsified or non-reproducible data to AI-generated citations, to insufficient documentation of methods, data, and code,” Rockenbach said. “These challenges matter not only for researchers, but for everyone who relies on science to inform medical research, public policy, and a shared understanding of the world.”
Rockenbach stressed how public trust in science remains fragile after the politicization of COVID-19 mitigation measures and debates about the robust research demonstrating the safety and efficacy of vaccines. She said there are no shortcuts.
“Reproducibility and rigor are not technical side issues,” she said. “They are core to scientific inquiry.”
Buck spoke at separate events moderated by Megan Ranney, dean of Yale School of Public Health, C.-E. A. Winslow Professor of Public Health (Health Policy), and professor of emergency medicine, and Michelle Hahn, deputy director and chief operating officer of the Tobin Center.
Buck discussed how as a vice president at the philanthropic research organization Arnold Ventures, he launched a project that could only replicate about 40% of a set of psychology studies. In a second project, researchers attempting to replicate 50 cancer studies from top journals could only complete 23 of them, taking twice as long and costing twice as much money.
“In literally every single case, they could not even try to redo this cancer biology study just based on the published paper’s methods,” he said, adding that the effects they could replicate from any of the studies were on average 85% lower than those reported in the original publication.
Ranney highlighted a recent review paper showing high reproducibility of studies using real-world health data.
Buck distinguished between trusting the scientific method and trusting individual scientists, who may be sloppy, biased, or unethical. He cited historical abuses — such as eugenics research and the decades-long Tuskegee experiment to test the effects of untreated syphilis — to explain why skepticism is not necessarily irrational.
But he emphasized how the two cancer labs’ experience with the differing stir rates for an enzyme solution revealed how replication difficulties do not necessarily stem from incompetence but from how much of scientific practice can rely on hidden, undocumented, tacit knowledge.
He argued that scientists should resist overselling their results and focus less on percentage estimates and more on a range of possible outcomes that could better help politicians shape policy. He and Megan Ranney discussed how researchers should better explain uncertainty and the evolving nature of science, even as political actors might seize on expressions of doubt to advance their own agenda.
“There’s something there about the way science happens that feels like maybe we’ve not been having the right conversation with the public,” Ranney said. “Even when we try to offer nuance, it often gets erased in the reporting or in the two‑minute cable news hit.”
Buck and Michelle Hahn discussed how scientists and politicians often seek different types of evidence and operate on different timelines.
“Policymakers are responding to immediate pressures and opportunities,” Hahn said. “While good science takes time to produce clear, credible evidence. The challenge is aligning those timelines without compromising rigor.”
Buck endorsed pre-analysis plans as described by David Yokum of the University of North Carolina, in which policymakers would agree in advance how they would respond to an experimental outcome.
He also discussed how grants, academic tenure, and publication reviews should reward replication, null results, and rigor. Scientists have little incentive to publish non‑significant or negative results, distorting scientific literature to show strong effects, skewing effect sizes, wasting resources on experiments already shown unpromising or doomed, and undermining evidence-based medicine and policy.
“As far as institutions, my advice to agencies like NSF and NIH — they have close to $60 billion a year to play with,” Buck said, adding that they should cut counterproductive bureaucracy, streamlining administrative work to free up more time for research. “Those agencies should pay people to do more replications or offer more money for data sharing. Give preferential treatment. That could definitely change practices over time.”
Buck sees some AI tools as breathtaking advances but also fears an arms race of AI-generated manuscripts and AI-generated reviews. He cited early evidence that AI may help researchers produce more papers but also narrow their intellectual range — a potentially dangerous tradeoff.
He called for sharing code in public repositories and for deploying preregistration of experiments as a valuable tool to prevent “garden of forking paths” exploration and undisclosed hypothesis‑shifting.
In addition, Buck said that studies should contain enough subjects with enough statistical power to produce statistically significant results applicable to wider populations. And that researchers should document their work more thoroughly and share data, methods, and results openly and often.
He insisted that the future of credible science will depend on institutional incentives, not just better individual intentions. And on garnering the confidence of a more educated public.
“Every middle school science teacher in America should be telling students why you shouldn’t trust a single mouse study,” Buck said. “And what are the indicators of trustworthiness.”
Ron Borzekwoski, executive director of DISSC, expressed his appreciation for Buck’s work and touted DISSC’s commitment toward the same goals. DISSC promotes open and reproducible research by providing infrastructure, centralized resources, and expert consultation through its Open and Reproducible Research Program, led by ISPS Associate Director for Research & Strategic Initiatives Limor Peer.
“We are grateful for Stuart’s clear and candid discussion of the challenges and opportunities for science today,” Borzekowski said. “His personal history, institutional critiques, and experience from inside academia, philanthropy, and government show us how science can sometimes be less trustworthy than it should be. But, most importantly, that we can work together to improve.”