A survey published in the science journal Nature reports there is a crisis in the reproducibility of research results across several fields of scientific inquiry. The report, published in the May 26th, 2016 issue of the journal, states that “70% of researchers have tried and failed to reproduce another scientist’s experiments, and more than half have failed to reproduce their own experiments”. The survey, conducted among over 1,500 scientists at the coalface of research, echoes previous studies and surveys suggesting a widespread problem in the conduct of research.
The survey’s results showed that over 60% of researchers working in earth sciences and over 85% of researchers working in chemistry had failed to reproduce other scientists’ work. In the fields of medicine and biology the figure was in the region of 70% of scientists reporting an inability to reproduce their peer’s experimental results. Reasons cited for a failure to reproduce results included issues around selective reporting in published papers, pressure among scientists to publish, poor statistical analysis and low statistical power, insufficient oversight/mentoring, poor experimental design, fraud and insufficient peer review.
In terms of addressing the reproducibility crisis highlighted by the survey, almost 90% of respondents reported that more robust experimental design, better statistics and better mentorship would go a long way to improving the reproducibility of research work. In a lead editorial of the same issue, the journal highlighted that over two thirds of scientists surveyed felt that there was a “major problem” in reproducing research results and warned that “it would be foolish to pretend that there is not serious concern”. While the editorial recognized a certain inevitability that some scientific research will not be reproducible, it was also acknowledged that the problem is bigger than this alone. The problem will be of concern to funders, publishers and policy makers who rely on the scientific literature to direct investment in research and development and to make decisions that affect significant proportions of the population. Efforts to improve reproducibility and protect research to ensure robustness in published findings will attract increasing interest as the research community determines the best path forward.