Written Evidence Submitted by

Marcus Munafò, Chair of the UK Reproducibility Network Steering Group

(C190063)

 

The UK Reproducibility Network (UKRN; www.ukrn.org) a national peer-led consortium that aims to ensure the UK retains its place as a centre for world-leading research. We do this by investigating the factors that contribute to robust research, promoting training activities, and disseminating best practice. We also work collaboratively with various external stakeholders to ensure coordination of efforts across the sector.

 

This submission relates to the following terms of reference:

 

2. The capacity and capability of the UK research base in providing a response to the outbreak

3. The flexibility and agility of institutions and processes to respond on the above during a crisis

 

  1. The SARS-CoV-2 / COVID-19 pandemic has resulted in an unprecedented diversion of research activity. Much of this has been stimulated by rapid funding calls by research councils and charities. However, there are concerns among the academic community that research activity has been excessively “covidised” 1, and that this brings negative unintended consequences.

 

  1. First, this diversion of research activity is at the expense of research into other global biomedical and societal challenges that remain pressing. For example, more articles have been published on COVID (32,850 results in PubMed between 2019 and 2020) than for “sickle cell” (28,845 results between 1923 and 2020) or “anorexia nervosa” (16,488 results between 1911 and 2020) across the entire corpus, as of 23rd July 2020. At the same time, the Gates Foundation has diverted its resources to COVID at the expense of supporting research capacity in low- and middle-income countries 2.

 

  1. Second, the academic community has also raised concerns that the speed with which evidence is being produced is at the expense of quality 3. UKRN recently published a position statement emphasising the need to ensure that even when there is a pressing need to understand a new phenomenon rapidly, researchers must still adhere to the fundamentals of good design, careful conduct and thoughtful interpretation 4. We have already seen a number of high-profile retractions, but perhaps a greater concern is the proliferation of studies that do not genuinely advance efforts to tackle the pandemic.

 

  1. This illustrates a third challenge: that effective exploitation of research requires mechanisms to curate, summarise and aggregate findings from many (often thousands of) studies. The work of the US Librarian Reserve Corps, the COVID Living Systematic Map of Evidence at EPPI-Centre/UCL, and the COVID Systematic Online Living Evidence Summary at the University of Edinburgh all show what can be achieved, but these (and other similar) efforts have been ad hoc, only loosely coordinated, and necessarily delivered in haste. We require capacity to routinely curate, summarise and aggregate research findings.

 

  1. A fourth issue is that rapid funding calls place the onus on researchers to identify the most important research question, which runs the risk that researchers will identify questions that simply align with their existing activity, given the strong incentive that the availability of funding creates. Arguably, a situation like the current pandemic requires greater top-down direction than is usual in academia, so that key uncertainties can be resolved. Funders, working with structures such as SAGE, could identify key questions and release targeted calls for proposals to answer those questions.

 

  1. UKRI recently released a report on incentives in research 5. In our view, the pandemic has amplified problematic incentives (e.g., to publish and to obtain funding) in a way that has not helped our efforts to respond effectively to the pandemic and build a robust evidence base. It is critical to learn from this experience, both in the context of the general role of incentives, and their specific impact during an acute event such as the pandemic. We cannot “follow the science” if it turns out that the science is simply following the money. Again, greater top-down direction in future similar situations could address this.

 

  1. Finally, in order to maximise efficiency and minimise publication bias, we also recommend that funders award such projects using the Registered Reports article type. Under this model, the funder and a peer-reviewed journal assess proposals simultaneously, with the article pre-accepted at the same time as funding is awarded and, crucially, regardless of the eventual results. This approach eliminates both publication and reporting bias, and can further benefit from the rapid review system for Registered Reports launched by the Royal Society in response to the pandemic 6.

 

Summary of recommendations

 

  1. Establish the infrastructure to routinely curate, summarise and aggregate research findings, ensuring this is agile enough to respond to evidence emerging rapidly during an acute scenario such as a pandemic.

 

  1. Ensure that rapid funding calls and other calls for information include a degree of top-down coordination to ensure that key questions are answered and key uncertainties resolved.

 

  1. Explore whether rapid funding calls could be integrated with Registered Reports article types (known as Registered Reports funding partnerships 7), to ensure rapid publication on completion and prevent publication and reporting bias.

 

References

 

  1. ‘Covidisation’ of academic research: Opportunities and risks. https://naturemicrobiologycommunity.nature.com/posts/65638-covidisation-of-academic-research-opportunities-and-risks
  2. Gates Foundation to concentrate on coronavirus. https://www.ft.com/content/f4557f2c-2464-46bd-a844-d08cadc3da59
  3. Research on COVID-19 is “suffering from imperfect incentives at every stage”. https://www.bmj.com/content/369/bmj.m2045
  4. UKRN Position on COVID-19 Research. https://www.ukrn.org/2020/05/01/ukrn-position-on-covid-19-research/
  5. Research Integrity: A landscape study. https://www.ukri.org/files/legacy/documents/research-integrity-main-report/
  6. Urgent call for Registered Reports on coronavirus. https://royalsociety.org/blog/2020/03/urgent-call-for-registered-reports-on-coronavirus/
  7. Improving the efficient of grant and journal review: Registered Reports funding. https://academic.oup.com/ntr/article/19/7/773/3106460

(July 2020)