Written Evidence Submitted by Taylor and Francis Group

(RRE0023)

Introduction

Taylor & Francis Group is an international academic publisher, publishing more than 2,700 journals and over 5,000 new books each year, with a books backlist in excess of 120,000 specialist titles.

Supporting and fostering research integrity is a priority for Taylor & Francis. Alongside other scholarly publishers, we are actively engaged in developing standards and best practice to support the responsible conduct of research and the publication of quality, reliable research outcomes across the disciplinary spectrum. We work with hundreds of UK-based learned societies and professional member associations to support the publication of trusted and high-quality research outputs which represent the best outcomes from their communities, drive consensus, and advance discourse and developments within their specialised fields.

We believe that research outcomes are most impactful when they are open and have resulted from a transparent research process, and are committed to driving and supporting Open Research. In support of this, we offer a broad range of open access publishing options, including fully open access gateways - including the European Commission’s Open Research Europe platform through our innovative imprint, F1000 - as well as a selection of journals and books across the disciplinary spectrum.

We hope that our inputs to this Call for Evidence will be useful and we would welcome the opportunity to further support the Committee on this issue by developing practical solutions to help resolve any challenges. 

Responses

1. The breadth of the reproducibility crisis and what research areas it is most prevalent in

We understand reproducibility to cover a broad range of themes, including – but not limited to – computational reproducibility, replication, and the ability for researchers to build on previously published work with faith that it meets high ethical and experimental standards[1]. As such, we encourage the Committee to consider reproducibility in its broadest understanding, based on the foundational premise that all published research findings, irrespective of discipline, should be verifiable, replicable, and credible. Even in the Humanities, where replication may not be considered as valuable a concept, there should still be accountability for research design and potential value from increased trust and impact[2].

This broad scope poses some challenges, most notably the lack of any ‘one size fits all’ approach, given the breadth of research methodologies, study designs, formats, and definitions across disciplines. However, we believe that this is much to be positive about. Firstly, actors from across the scholarly communications ecosystem are constantly evolving processes and practices which help to self-correct the record of science, supported by better technologies. Secondly, we are seeing a shift in values within the researcher community itself to improve research integrity, to act with greater transparency and integrity, and to collaborate to find global solutions and standards that will foster efficiency and bring about change. We expand on some of these initiatives in our response.

2. The issues in academia that have led to the reproducibility crisis

If we were to highlight only one cause of issues around reproducibility, it would be the research system itself, which is complex, fragmented, and not optimised to make research outcomes reusable and reproducible. To combat this, all stakeholders have a role to play in establishing a more holistic approach to the research and publishing process. We signpost some promising initiatives and suggest avenues for further development in our response, covering both values-based and operational initiatives.

Causes include but are not limited to:

• Rewards and incentives that drive the publication of novel research outcomes, often at the expense of ‘sound science’ (including incremental, null and negative results). This can particularly impact early career researchers who need to establish a research output portfolio to facilitate a career in academia, and who often subsist on precarious employment contracts linked to research assessment.[3] Similarly, rewards and incentives are rarely tied to activities (such as peer review) which reinforce integrity and trust in published research outcomes.
• The ‘publish or perish’ culture is thought to be responsible for introducing considerable waste into the research system; many research findings are unpublished and not easily discoverable, or results are duplicated by researchers in similar fields. There are many estimates that demonstrate[4] the huge potential research waste in the system caused in part by the way research is shared and published. 
• Systemic bias in who produces and publishes research – the research and scholarly communication communities do not reflect broader society. This lack of diversity can lead to blind spots, such as research outcomes which focus on one demographic to the detriment of other groups[5].

• Limited systems and processes that make research reproducible and trust-worthy –

there is much progress still to be made across the whole research system that could ensure that research methods, data and all outputs are shared, described and discoverable and accessible in ways that maximise their potential to be used and useful.  As a scholarly publisher, we are working hard to make the research we publish  as ‘FAIR’ (Findable, Accessible, Interoperable, and Reproducible) as possible, but to do this requires the research to be produced in formats and with the necessary detail and descriptors to make this possible – and this needs coordinated input and guidance from funders and research institutions – and support and training for researchers /authors.

3. The role of the following in addressing the reproducibility crisis

As a scholarly publisher, we are involved in and/or supportive of a range of initiatives that have emerged to help address many of the challenges and opportunities that we now face to make research reproducible and to assure trust and integrity. We have included a table with illustrative examples of key current initiatives designed to address this issue which we hope will be useful to the Committee.

4. What policies or schemes could have a positive impact on academia’s approach to reproducible research

Researchers globally look to the UK for leadership on all areas of research and development, recognising the quality of our published research outputs, and the stringent frameworks already in place for ensuring the accuracy and integrity of our research practices. We believe that the adoption and endorsement of additional programmes and initiatives by the UK Government (about which more below) offers the opportunity for the UK to expand its reputation and leadership around research excellence. We have identified four key areas where policy intervention could positively influence and improve the reproducibility of research. These are open research (science), training, rewards and incentives, and global standards. We offer some suggestions below around specific areas for policy input.

Open research

Key recommendation: investing in appropriate and interoperable infrastructure.

There is global movement towards more open and collaborative ways of working across science and research more broadly (‘open research’), the aim of which is to democratise access to knowledge and enable collaboration in ways that accelerate the potential for research to have impact. Open research encourages researchers to share the products and outputs of research; including throughout the research lifecycle e.g. from study design and methodology, and research data and software, in addition to negative and null findings, to support robustness and transparency in the way research is conducted, thereby enabling trust and transparency in research. We note that UKRI has already made significant strides in promoting Open Research and the Committee would be strongly placed to use its influence to further support and incentivise Open Research activities as a means to also improve the discoverability, usability and reproducibility of research and its integrity – specifically through ensuring funding is available to introduce and develop supporting infrastructure (e.g. persistent identifiers).

Training

Key recommendation: targeted investment in training around data science skills, including analysis, curation, and formatting to enhance the reproducibility of research outcomes across the disciplinary spectrum.

Many researchers are willing to share their data, but lack confidence in data skills, especially ensuring that their data is FAIR (Findable, Accessible, Interoperable, and Reproducible). Research Offices at institutions could be provided with dedicated funding for training around responsible research conduct and data stewardship. Support could also be directed to community-led groups (for example the Research Data Alliance and its Interest Group on Data Stewardship).

Alignment with global standards and best practice

Key recommendation: evaluate and align with existing initiatives; formally adopt or endorse principles.

The global scholarly communication and research communities have developed a significant body of guidelines and best practices that apply to general research practice and communication, as well as to specific fields (many of these are outlined in question 3 above). These resources have been created to foster quality, reliability, and rigour in research and the outcomes of the research process. Many of the guidelines developed have gained significant support within the community as they outline common and interoperable standards, promote transparency, outline common methodologies and checklists, and provide tried and tested examples of best practice that are valued by researchers. These have typically been developed with the input of key stakeholders including researchers, learned societies, funders, institutions, and publishers.

Rewards and incentives

Key recommendation: revise rewards and incentives structures in the next REF to reward those who support and further research integrity.

Research integrity often relies upon invisible work carried out by many researchers in the service of their communities – for example peer review, mentoring, training colleagues, taking up advocacy roles within their institutions.  We applaud UKRI for its focus on this area, including advocacy around the narrative CV. Good progress is already being made, but there is an opportunity to go further, faster.

The upcoming evaluation of the Research Excellence Framework (REF) is an ideal opportunity to revise incentives structures to reward research integrity and the responsible conduct of research. The Hidden REF may be a useful source of inspiration.

5. How establishing a national committee on research integrity under UKRI could impact the reproducibility crisis

Key recommendation: ensure that any committee include broad representation.

We suggest that any national committee on research integrity aims to align with global initiatives, to work with all stakeholders, and to adhere to standards that have been developed or are evolving as the result of cross-stakeholder inputs (see response to the question above for some examples). The national committee may find that it shares many aims with the UK Reproducibility Network (UKRN).

For the committee to be able to make evidence-based interventions that benefit all, we suggest that it has broad representation to ensure that its direction can be informed by a range of views. This diverse representation should cover academia (including cross disciplinary, early career, minority ethnic, disabled, and other marginalised groups), as well as including other stakeholders in the research sector (institutions, funders, industry, publishers and service providers).

In our experience, policy measures that incentivise and reward researchers are more effective in the longer term than prescriptive or punitive measures (such as sanctions). We have found that policy interventions also have more success when they are supported by appropriate resourcing, be that funding, training, or guidance. For these reasons, we believe any national committee i) needs to have broad representation, ii) should cover the disciplinary spectrum, and iii) should ensure that any policy direction is supported by appropriate resources and funding. We also advise that the committee iv) reviews existing best practice and standards, with an aim to align with these wherever possible to facilitate improvements to global research culture.

(September 2021)