Written Evidence Submitted by Responsible Research in Practice

(RRE0081)

What follows is a submission written by Dr Nikki Osborne and submitted on behalf of Responsible Research in Practice Ltd. It contains opinions based upon 25+ years personal experience working as a researcher, ethical and peer reviewer, trainer and consultant within the Life Sciences research sector.  All comments relate to in vivo and in vitro Life Science research only and are shared to contribute to discussions regarding the interplay between research integrity and issues of scientific rigour and reproducibility.

About Responsible Research in Practice Ltd

Responsible Research in Practice is a UK company set up in 2015 by Dr Nikki Osborne. The intention at the time was two-fold; (1) to put good animal welfare and implementation of the 3Rs in the broader context of responsible research conduct; (2) to increase awareness of research integrity and reproducibility issues within the laboratory animal sciences. Both goals have been pursued through the provision of specialist training courses tailored to the national research framework and to address in-house concerns within client Life science research organisations. Since 2015, the main change has been a broadening of the company’s scope outside of the laboratory animal sciences/ in vivo research. There has been much progress made within the Life Science research sector over recent years in terms of the development of alternative non-animal research methods and models. In response to this the company has expanded the remit of its training and free responsible research webinar series to include in vitro research. Thus, Responsible Research in Practice provides the Life Sciences research sector with specialist training and consultancy services focussing on promoting best practice to fulfil expectations of responsible research conduct, research integrity, and to improve rigour and reproducibility.

About Dr Nikki Osborne

Nikki’s research career began in 1996 when she was awarded a BSc in Biomedical Science (Neuroscience) from Kings College London. She immediately began working in an experimental pathology laboratory at Kings before starting her PhD in developmental neurobiology. Nikki was awarded her PhD in 2004 before taking up a postdoctoral researcher position within the Ludwig Institute for Cancer Research (UCL branch). After the 2005 London bombings Nikki decided to leave London and in 2006 took up a scientific advisory role as a biotechnology specialist within the RSPCA. Here she stayed for 10 years before deciding to set up her own company – Responsible Research in Practice Ltd. During her career Nikki has accumulated over 25 years of experience working within the life sciences, animal research and animal welfare community’s across the UK, Europe, USA, Canada and beyond. She has worked with academic, commercial, regulatory, and non-governmental organisations to promote responsible research conduct, animal welfare and the 3Rs principles of humane science, as well as raising awareness of research integrity issues including poor rigour and reproducibility, and research reporting standards.

In addition to her role as company director and lead tutor at Responsible Research in Practice Nikki remains an active member of the Life Science research community promoting integrity, ethics, and responsible research practices. She is regularly invited to speak about improving the reproducibility of preclinical and animal research. Most recently this has included talks at the 11th World Congress on Alternatives and Animal use in the life sciences and Annual Conference of the Australian and New Zealand Laboratory Animal Association (ANZLAA). Nikki also remains an active member of two Animal Welfare and Ethical Review Bodies (AWERBS) here in the UK and is involved with several 3Rs committees including that of the ISTT (International Society of Transgenic Technologies). She is also a member of COPE (Committee on Publication Ethics), and an advisor to UKRIO (UK Research Integrity Office) on animal research topics.

Examples of efforts to improve research integrity and reproducibility

• Responsible Research in Practice is unique in the range of specialist professional training courses it offers focussing on ‘Responsible Animal Research’, ‘Improving Reproducibility’ and ‘Statistical Analysis Methodology’. We also work with experts in systematic reviews, and experimental design to deliver bespoke training courses for client organisations (primarily UK universities and research organisations).
• Responsible research in practice runs a free monthly responsible research webinar series that currently attracts attendees from across the UK and another 54 countries around the world. The purpose of this webinar series is to improve awareness of rigour and reproducibility issues within the life sciences research sector, and to inspire individuals to use freely available tools and resources to improve how they plan, design, conduct, supervise or manage their research activities. These webinar sessions complement and extend the topics discussed within our training courses.
• Consultancy projects completed include writing the UKRIO Research Integrity primer on research involving animals.
• In addition to talking on the topic of improving reproducibility within life science research, Company director Dr Nikki Osborne has authored many publications relating to improving research reporting standards, reproducibility and research integrity within the laboratory animals sciences (see ORCID Profile for more information).

● the issues in academia that have led to the reproducibility crisis;

There are variable levels of awareness and understanding of the causes of irreproducible research and the factors that are within the control of individual researchers. In our experience training Life Science researchers many are aware of reproducibility issues, but rarely do they relate this to their own research disciplines, or personal research conduct. Indeed, when we poll training and webinar participants, we find that the majority are aware of reproducibility problems but do not know what they can personally do to work towards resolving them. Attendees of our courses tell us they have become inspired and confident to lead by example following the training and support we provide to enable them to review and change their own practice.

Researchers are not routinely recognised or rewarded for improving the rigour and reproducibility of the research they conduct. This is important because researchers have many demands upon their time that mean that lifelong learning/CPD often falls down the priority list. Having a growth mindset and keeping up to date with best practice (it is by necessity an ever-evolving concept) are essential components that enable individuals to think critically, challenge common practices and change their behaviour. This is critical to efforts to improve the rigour and reproducibility of the research they conduct. The benefits of this are clear, and yet these qualities indicative of excellence are not recognised, rewarded or celebrated within the scientific community.

Inconsistent training in how to plan, design, conduct, manage, supervise, analyse and report research. Many academic establishments provide very basic level generic e-learning training on research integrity and reproducibility issues that restates expectations that can be applied across all subject areas. In reality, there are numerous practical differences in how STEM research, and arts, humanities and social sciences research is planned, conducted and the research outputs generated. Because of this generic research integrity and reproducibility training is ineffective. It fails to contextualise information so whilst researcher may understand the general principles, they remain unable to recognise what this means in the practical context of their own research activities and unable to develop the skills required to apply the principles during the course of their work.

● the role of the following in addressing the reproducibility crisis:

- research funders, including public funding bodies;

We recognise that UKRI funders and others (Wellcome Trust, Royal Society, Nuffield Council on Bioethics) have been working together to identify and seek to improve issues of rigour and reproducibility for many years. Examples of this include: the AMRC “Reproducibility and reliability of biomedical research: improving research practice” symposium in 2015; NC3Rs experimental design assistant tool and ARRIVE reporting guidelines; Nuffield Council on Bioethics 2014 report “The Culture Of Scientific Research In The UK”; Wellcome Trust 2020 report “What researchers think about the culture they work in”. We welcome these efforts but believe that research funders can do more.

We would like to see funders provide a list of key qualities funding applicants need to demonstrate evidence of when applying for funding and for specific training requirements to be linked to individual approved funding applications. For example, a growth mindset and critical thinking skills are key qualities. Plus all researchers should receive regular training in how to plan, design and conduct research in accordance with best practice as it relates to their personal research discipline and the experimental techniques and methodologies used. Funders currently require successful applicants to provide evidence of the outputs resulting from the funded project. We would therefore like to see funders require applicants to upload evidence of specialist training attended and other ongoing CPD activities that relate to the delivery of funded research in accordance with best practice. This would emphasise the importance attached to lifelong learning and it an indicator of the presence of a growth mindset. It is not sufficient in our view for funders to provide generic expectations of researchers and/or host institutions/organisations with regards training.

We would also like to see research funders proactively identifying and showcasing the efforts of the researchers they fund to improve rigour and reproducibility. Funders have the potential to identify individuals using the project outcome information collected using the ResearchFish system and highlight them using the UKRI Gateway to research portal. Such an approach illustrates what is possible within different research disciplines, emphasises the benefits for science and Society and is consistent with the responsible research and innovation agenda.

We would like to see funders encourage greater use of systematic review and/or meta-analysis methodology. By conducting new analyses using existing data potential uncontrolled variables and/or confounding factors can be identified. The effect of these critical variables/factors can then be minimised during future experiments using a factorial experimental design and resulting in more reproducible results. Equally novel meta-analyses of existing data can on occasion partially or completely replace the need for additional experiments to generate additional data. This is about maximising the benefits arising from existing data/experiments so is about greater utilisation of existing research outputs rather than blindly funding new experiments to generate new data for analysis.

We would like to see funders set up a funding scheme to enable researchers to apply for funding to cover the costs for specialist training courses. We envisage that funders could either select organisations to partner with to identify specialist courses that they feel researchers would benefit from attending, or they could have a dedicated funding call so that researchers identify the specific training course they wish to attend and apply to have the registration fees funded as required. We are aware that researchers can include costs for training within their grant applications but in our experience, funds provided for training costs are often spent on other research costs not covered by the grants. This is another reason why we would like to see research funders require applicants to upload evidence of training to demonstrate efforts to develop or update core skills and key qualities.

- research institutions and groups;

At present there is a lot of internal politics within academic research institutions regarding the funding of training for researchers. Often generic training qualifies for core funding whilst training that is discipline, technique or methodology specific is not and so does not happen. We believe that research institutions should pride themselves on providing specific training and support that enables staff and students to implement good practice. Excellent, rigorous and reproducible research does not happen by chance, it requires requires individuals to undertake and achieve three stages of learning:

• Stage 1 - an awareness and understanding of contemporary best practice standards in all aspects of research practice (including ethics, integrity, experimental design, methodology, statistical analysis and research dissemination). This training needs to tailored to specific research disciplines, methodologies or techniques.
• Stage 2 - the ability to recognise what 'best practice' means for a given research activity or project at a given moment of time. Note - this will change throughout a researcher’s career as they move labs and take on additional roles such as supervisor, peer reviewer, manager, lecturer, mentor. It therefore needs to be tailored to individual establishments, disciplines, and career stages.
• Stage 3 - the confidence and competence to implement best practice and deliver rigorous, reproducible research when it matters i.e on a daily basis.

We do not believe that all three of these learning stages are currently recognised and catered for. There are many reasons for this:

• There are too few places currently available on specialist training courses to address this need. The UK is world leading in the kinds of specialist training available via organisations such as the UK Reproducibility Network and companies such as ourselves, but more needs to be done to increase access to this training and the numbers of researcher benefiting from these courses.
• There are also cultural issues limiting the accessibility and availability of specialist training because traditionally stages 2 and 3 were attained through working alongside peers and colleagues. In the current research culture this is not achievable for several reasons including:

–         PhD students and junior researchers are required to work independently at a much earlier stage without dedicated training or mentoring. This is because their bosses, peers and more experienced colleagues are often too busy to be able to spend time with them unless they have results to discuss and are not acknowledged or rewarded for doing so.

–         Older researchers who are commonly the ones deciding on training provision for junior members of their teams learnt how to conduct research by the traditional method of working alongside others. They therefore assume that the new generation will learn the same way and so do not see the need or value of providing specific training in discipline specific core skills.

–         Many common research practices relating to the planning, design, conduct and analysis of research are no longer good practice, but they have become entrenched because researchers are too time poor to challenge or critique them. Thus what little students and junior researcher can learn from their peers is often poor practices that further contribute to reproducibility issues. Examples include: using the research model/method they are most familiar with rather than the most appropriate to generate or test the hypothesis; not implementing approaches to minimise potential sources of bias such as randomisation & blinding; incorrectly identifying the experimental unit, using a sample size based upon experience rather than a power calculation; not deciding upon the method of statistical analysis in advance; not understanding the difference between correlation and causation, not defining inclusion/exclusion criteria, not designing experiments to take into account potential uncontrolled variables and confounding factors.

- individual researchers;

All the researchers that we work with are passionate about their work, want to be the best researchers that they can, and conduct their research in the best way they know. This is exemplified by the several early-career researcher lead initiatives seeking to improve rigour and reproducibility. For example:

–         ReproducibiliTea is a volunteer run journal club initiative that started in Oxford in 2018 for early career researchers to discuss research papers, ideas about improving science, reproducibility and the concept of ‘open science’ more generally.

–         The RIOT Science Club is a seminar series that started at Kings College London in 2018 with the purpose of raising awareness of and provide training in Reproducible, Interpretable, Open & Transparent science practices.

–         The UK Reproducibility Network is a grass roots, peer-led organisation aiming to investigate the factors that contribute to robust research, promote training activities, disseminate best practice, and work with stakeholders (researchers, institutions, publishers and funders) to coordinate efforts across the biomedical sciences and other research disciplines to promote robust and rigorous research practices.

In our experience attendees of our training often take it upon themselves to act as local role models to inspire others working around them to change their research practices to improve rigour and reproducibility. This amplifies the impact of training individuals, but unfortunately we do not believe that all researchers have equal access to the training and support they need to be able to keep up to date with best practice and conduct rigorous, reproducible research. The reasons for this are outlined above and we believe that this is a missed opportunity to utilise this passion to enhance ongoing efforts to drive behavioural change.

In summary, there are initiatives, tools and resources available to researchers working within the Life Sciences sector to improve rigour and reproducibility. However, uptake and use of these freely available tools is currently limited for several reasons:

• Many academic researchers do not realise that many well-established common practices are in fact poor practices contributing to poor rigour and reproducibility;
• The uptake and use of tools and resources to improve rigour and reproducibility is not as commonly promoted or widespread as entrenched poor practices;
• Access across the UK to specific practical training in aspects relating to the planning, design, conduct, analysis and dissemination of research is patchy.
• Critical, challenging and creative thinking, and a growth mindset are not core skills or key qualities being routinely developed or encouraged within the research community;
• Often researchers do not recognise that they are contributing to reproducibility issues, or that there are actions they can take ownership of and improve the rigour and reproducibility of the research they conduct.

(September 2021)