MPs call for research sector reforms to address concerns with reproducibility of science

10 May 2023

A new report by the Commons Science, Innovation and Technology Select Committee highlights concerns over the reproducibility of scientific research.

The report notes that despite the Government's increased focus on research and innovation and the largest-ever increase in public investment in research and development, the integrity of some scientific research has been called into question because of difficulties in reproducing claimed findings of experiments or analyses of data. This has led to concerns over what some have called a 'reproducibility crisis'.

The Committee welcomes the establishment of a new Committee on Research Integrity, which was a recommendation of the predecessor Committee’s 2018 report. However, it expresses concern about the absence of reproducibility as a priority in the new organisation's strategy.

The report emphasises the need for a comprehensive and rigorous assessment of the scale of the problem in the UK and which disciplines are most affected, and in particular the impact of deploying Artificial Intelligence—and other increasingly complex software—on reproducibility in research.

The report recommends the establishment of a sub-committee of the Committee on Research Integrity focused solely on questions of reproducibility in research.

The Select Committee’s report addresses recommendations to funders, including UK Research and Innovation. It says funders should consider whether grants awarded provide the resources necessary to ensure that work they fund is reproducible.

The report recommends that funders should require reproducibility as a condition of grants awarded, as well as training researchers in research integrity and the need to ensure reproducibility. The report also backs the wider use of the 'resume for researchers' format in funding calls, including evidence of commitment to reproducibility.

The report calls on publishers of academic journals to ensure that there is no bias towards publishing studies whose findings are novel over those whose results confirm previous studies. It also calls on publishers to commit to more timely publication of retractions of published articles and corrections of errors.

Committee Chair Rt Hon Greg Clark MP said:

“Scientific progress requires transparency: being able to reproduce analysis to check research findings are robust.

“With the annual publicly-funded R&D budget set to reach £20 billion next year, it is especially important that findings stand up to scrutiny.

“But there are concerns that many scientific findings are difficult or impossible to replicate, with AI posing new challenges to transparency.

“We call for a clearer focus from research funders, publishers and academic institutions to ensure that research is transparent and reproducible.”

Key recommendations:

Reproducibility as a condition of grants awarded for empirical research
Review the Research Excellence Framework assessment criteria to assure that transparency is a prerequisite of top-scoring research.
Publishers should review their record of publishing confirmatory studies
Publishers should commit to more timely publication of retractions of published articles and the correction of errors no more than two months after they become known
Require, outside exceptional circumstances, the deposition of research data and code in open-access repositories alongside the publication of research outputs
A coordinated policy on minimum protected research time for research staff
A three-year minimum contract for post-doctoral researchers in universities
Mandatory reproducibility training for researchers at undergraduate, postgraduate, and early career researcher stages
A pilot programme to fund replication studies
Dedicated funding for the presence of statistical experts and software developers in research teams
Develop a ‘registered report’ model in which funding proposals and prospective publication are peer-reviewed before results are obtained

Recommendations to the UK Committee of Research Integrity:

Publish an annual state of the nation report with action plans and timings for implementing measures
Establish a sub-committee focused solely on questions of reproducibility in research
Investigate the impact of deploying AI—and other increasingly complex software—on reproducibility in research

Further information

Image: Pixabay