A new report on reproducibility begins with a lighthearted exchange between doctors in the 1973 Woody Allen movie Sleeper that foods believed to prolong life (wheat germ, organic honey and tiger’s milk) have been supplanted by steak, cream pies and deep-fat. “Those were thought to be unhealthy…precisely the opposite of what we now know to be true,” one says to the other. “Incredible!” the doctor responds.
It’s a scene that has stuck with Harvey Fineberg, M.D., who penned the preface to the new report, Reproducibility and Replicability in Science, published April 26 by the National Academies of Sciences, Engineering, and Medicine (NAS). In 2017, Congress mandated that the National Science Foundation (NSF) undertake the study.
Fineberg, president of the Gordon and Betty Moore Foundation since 2015 and chair of the committee that wrote the report, told RRC the exchange encapsulates the fallacy that “whatever we think we know now must be true.”
In reality, said Fineberg, former president of the Institute of Medicine, who also was provost of Harvard University and dean of its medical school, pointed out that “science, by its nature, gains deeper and sometimes quite altered understanding” with more research and “the fullness of time.”
That is not to say that there aren’t problems today of lack of rigor, reproducibility and replicability plaguing science—there are. But Fineberg and the committee members don’t believe there is a “crisis” at hand and are using the new report to, in part, explain the intrinsic iterative nature of research findings; their recommendations are offered in this context.
“One of the main ideas that we want to emphasize [is that] our committee found there is no crisis but also [there is] no time for complacency with respect to reproducibility and replicability in science. Improvements are needed,” he said.
The report offers a set of recommendations aimed at boosting “rigor and transparency in scientific research,” and proposes standard definitions of reproducibility and replicability, with the latter being among the strengths of the committee’s work, according to Fineberg (“Highlights From Reproducibility and Replicability in Science,” RRC 16, no. 6).