Written Evidence Submitted by Mr Christopher Marc Taylor, Chair, ISRCTN Registry

(RRE0050)

I submit the following evidence on a personal basis.

I chair the ISRCTN registry[1]. As a member of the international network of clinical trial registries coordinated by the WHO, the ISRCTN registry is recognised as the primary registry for the UK. I described the role and history of the ISRCTN registry in written evidence reference C190077 to your Committee’s inquiry “UK Science, Research and Technology Capability and Influence in Global Disease Outbreaks”.

This note is to emphasise the overlap between the issues raised by that inquiry and the Committee’s present inquiry into Reproducibility and Research Integrity. International pressures shape the UK research environment, including pressures affecting the reproducibility of research findings.

For reproducibility, the significance of the international system of clinical trial registries is that they share a standard template for describing and categorising clinical trials which, together with a growing consensus on sharing and re-using data, make it possible to treat findings and the data they are based on as a body of evidence. Systematic reviews consolidate this evidence so that successive clinical trials become mutually supportive and corrective. However, this settled international framework is for now limited to clinical trials and related health research designs.

For clinical research, US and EU law require registration on a public register and the timely reporting of results. The USA has this year begun to enforce the law on reporting, and the EU intends to do so when its Regulation on clinical trials comes into force in 2022. Campaigners such as AllTrials and Transparimed have become influential advocates of the public interest in transparency. Universities across the most research-active countries are finally responding to this pressure, and catching up with the levels of transparency that are now general practice in commercial health research.

In the UK, the Health Research Authority is implementing the transparency strategy Make It Public which it developed through extensive consultation in response to your Committee’s 2018 report on research integrity and clinical trials transparency. The ISRCTN registry is working with the HRA to prepare for the registration of all HRA-approved clinical trials from 2022. The HRA recently announced that it has begun to require a report one year after the end date of each clinical trial.

It is satisfying to see this movement towards transparency in clinical research gather momentum worldwide. But in the UK, I believe the Health Research Authority is still unique among public bodies in having as one of its statutory functions to promote the interests of those participants and potential participants and the general public by facilitating the conduct of research that is safe and ethical (including by promoting transparency in research).

Your Committee may wish to consider what statutory duties or functions would encourage the leading actors in other fields of research to create enforceable systems requiring transparency, responsible data sharing and reproducibility. At a time when research using big data is becoming ever more important and contentious, your Committee might well ask where to look for the impetus towards appropriate standard practices and reporting templates that would normalise transparency and reproducibility in data-driven research across all sectors. Qualitative research poses even bigger challenges.

Evidence from others will describe the breadth of the reproducibility crisis and the research areas it most affects. The focus of this note is on:

• the role of the following in addressing the reproducibility crisis:
• research funders, including public funding bodies;
• research institutions and groups;
• individual researchers;
• publishers; and
• Governments and the need for a unilateral response / action.
• what policies or schemes could have a positive impact on academia’s approach to reproducible research; and
• how establishing a national committee on research integrity under UKRI could impact the reproducibility crisis.

The main point of this note is to suggest that a significant response to the crisis of reproducibility requires concerted action from all the actors you listed, and that this concerted action won’t happen without leadership from both government and from respected individuals who can personify behaviour change and take sustained action to improve the culture around them. The UK’s culture of competitive excellence creates a research environment characterised by frequent and often unsuccessful bids for short-term funding. The pressure to seek new funding can easily take priority over finishing existing work to a high standard, or reporting outcomes in a contestable way. Time is scarce for quality control and the duties of sponsorship. The widely accepted belief that publisher favour novel findings can create a temptation to report selectively rather than building the evidence systematically.

The funding model results in extremely insecure working conditions for young researchers, creating pressures and moral dilemmas which seem highly likely to put at risk the quality of the science they work on. As graduate students they are encouraged to take up often highly significant questions and research them with inadequate time, resources or supervision. Then in the early years of their substantive careers they move from project to project, often recruited at the last minute to carry out studies they did not help to design. Insecurity makes them afraid of having their own ideas stolen or suppressed, and discourages the kind of teamwork which leads to self-critical science and well-documented reproducible findings. Has your Committee received evidence that junior researchers are experiencing a better research environment than they reported to the Nuffield Council on Bioethics in its 2014 survey of research culture[2]; and that improvements will survive the pressures on universities following the COVID pandemic?

With an understanding of the incentives and pressures in universities, it would be helpful if your Committee could give more weight in this inquiry

• to the public interest in seeing policy and regulation founded on reliable scientific evidence; and
• the mechanisms for international scientific collaboration which help to bring the full range of available research evidence to bear on issues where evidence is weak, stimulating new collaboration to strengthen the evidence base.   

It would be surprising if your Committee’s report had nothing to say about

• the influence of informed members of the public and campaign groups whose advocacy creates pressure for robust scientific advance, or
• the impressive advances in the quality of scientific journalism in the UK, or
• the place of reproducibility in international scientific efforts in which the Government says it aims to make the UK a leader. 

In the areas of clinical research which I know best, it is remarkable that medical charities for many years tolerated a high failure rate in research studies that affected their supporters, failing to chase up even studies funded by their own charitable efforts. But it is also remarkable how the public response to the COVID pandemic has revealed a powerful interest in the quality and completeness of scientists’ understanding, and in the way the government and other national bodies make use of the available evidence. The result is an impressive degree of public understanding and attention in the midst of a deluge of misinformation.

Even twenty years ago, the general level of numeracy and scientific awareness in the UK could not have supported this outcome. It would be a mistake now to regard reproducible science and research integrity as matters confined to unhurried debate among UK research institutions. Public understanding of research findings and public faith in government’s ability to identify and apply sound science now drive behaviour across sectors from climate and energy to health, to behaviour in public spaces. Science-related international comparisons now shape the way many people around the world judge the competence of their respective governments.

UK CORI

You ask, how could a national committee on research integrity under UKRI impact the reproducibility crisis? There are several reasons for wondering whether UK CORI could have any significant impact. First, the UKRI’s vision of the committee appears quite short-term, whereas the roots of the reproducibility crisis run deep. UK CORI has been set up with a life of three years. The first year will be consumed by recruiting its members and consulting on its priorities. Its remit is generally vague.

The UK CORI’s remit is precise only in relation to research misconduct: it will not have the authority to verify whether a research institution has followed appropriate processes to investigate misconduct. Instead, it is to provide a forum to discuss how standards and expectations for such processes can be set across the sector. If this becomes its focus it will tend to duplicate the activities of the existing UK Research Integrity Office without apparently having the means to reinforce what a subscription-based charity like UKRIO can achieve.

I suggest your Committee inquires whether there is any decisive evidence that the reproducibility crisis and related symptoms of weakness in the UK research environment are directly associated with misconduct of the kind a university would investigate. It seems more likely that reproducibility is undermined by complex incentives, leading to behaviours that fall short of misconduct but still inhibit collegiate effort within scientific disciplines and teamwork between them.

To take an illustration from the world of interventional health research, for reproducibility, it is crucial to design studies that have the statistical power to demonstrate clear findings and then to report the outcome in terms that are faithful to the design. The role of statisticians on the research team at key stages is therefore crucial. But it is unclear how many research institutions recruit, reward and celebrate statisticians in proportion to their importance. What is more, until recently it was commonplace for ethics committees to approve clinical research studies which later turned out to recruit too few participants to produce a statistically useful result. How many universities train their principal investigators sufficiently in the duties of sponsorship that they entrust to them?

Should we view failed clinical trials as honest errors or as the outcome of a research culture that shows a systematic neglect of the sponsor’s responsibilities towards individuals conducting research, towards volunteers who take part, and towards the public whose interest is damaged by crowding out more productive uses of scientific expertise and resources? The reproducibility crisis is in part a crisis of research productivity. The UK can’t afford to squander scientific resources or public trust.

The 2019 revision of the Concordat introduced a new core element to its definition of research integrity: ‘accountability’:

“accountability of funders, employers and researchers to collectively create a research environment in which individuals and organisations are empowered and enabled to own the research process. Those engaged with research must also ensure that individuals and organisations are held to account when behaviour falls short of the standards set by this concordat.”

Whereas this revision suggests the Concordat is meant to call for concerted action to improve the research environment, it is unclear from UKRI’s description of UK CORI whether anyone expects it to be the focus of such action. There is certainly no suggestion in UKRI’s description of it that it has been set up even with a consensus giving it a remit to coordinate robust efforts to tackle systematic issues across scientific disciplines, such as reproducibility. Instead, it might write a report.

(September 2021)