Page 4 of 4

Written Evidence Submitted by

David Gough, Professor of Evidence Informed Policy and Practice, EPPI-Centre, Social Research Institute, University College London

(C190097)

Executive summary

This submission addresses Terms of Reference:  6. The mechanisms for communication of scientific evidence internationally, within national governments and with the public.

There have been many critiques of government evidence advice used for the Covid-19 epidemic. The submission argues that the adequacy of the advice system requires: (i) an analysis of its aims, structures, processes and methods; and (ii) the use of ‘research on research use’ to appraise and develop these systems.

EPPI-Centre, UCL

I am an academic and Director of the EPPI-Centre at University College London.  The EPPI-Centre is a specialist centre for:

1. developing methods for the systematic mapping and synthesis of research evidence. We currently undertake a living (continually updated) map of the research on Covid-19 for the Department of Health and Social Care.
2. Research on research use: the study of how research is made use of in practice.

Introduction

The current pandemic has highlighted the important role of research to inform policy making. There have been many opinion pieces in the media critiquing both the science advice and its use in practice. The aim of the submission is to propose a more systematic approach to the analysis of the appropriateness of these advice systems.

How research evidence informs policy making is itself a topic of research study – ‘research on research use’ or the ‘science of using science’. This can provide such a systematic basis for studying and developing the current systems.

1. Governance systems and structures

There are a wide range of structures, processes and rules for identification, synthesis and interpretation of evidence in UK policy making. These include, for example, think tanks, What Works Centres, policy analysts, departmental research programmes, POST, enquiries by Parliamentary Select Committees, civil service posts such as the CSO, and science advisory committees.  Health is unusual in having a more integrated research and policy and practice system.

2. Specialist topic areas of science advisers

There are many types of evidence that might be relevant to policy making in response to a global pandemic. This creates issues for the selection of evidence topics. There have been critiques of the UK science advice system for not giving enough attention, at least initially, to public health, engineering, economics, and data on the seemingly effective precautionary strategies of some countries in the Far East. This raises issues of how topics are selected and by whom. To what extent is the topic focus driven by policy questions and how much by the scientific advice already available in the committees?

3. Science adviser roles and skills

The current discussions in relation to Covid-19 talk about science advice in terms of scientific experts providing evidence and advice in order to describe the size and nature of the problem and to estimate the relative impact of many possible interventions. This includes the many uncertainties and assumptions inherent in such advice.

Although ‘science advice’ sounds straightforward it can contain a number of different roles and responsibilities in the identification and use of evidence. For example, do all science advisers have the skills for the different roles of?:

• Identifying relevant types of evidence and research questions to ask
• Broad overview of a topic / nature of evidence and framing of a topic
• In depth technical detail of a topic
• Innovative ideas about what to study or what evidence might mean
• Challenge of assumptions of evidence claims
• Quality appraisal of evidence claims
• Synthesis and integration of different claims on same topic / selecting between different claims
• Integration of evidence across topics
• Interpretation and sense making of implications of evidence (including formal synthesis methods)
• Proposal of courses of action
• Identify likely outcomes and uncertainties in relation to different courses of action proposed by policy makers.

4. Identification and selection of science advisers

Even within topic areas science advisers are selected in some way.  This raises questions about:

• How a potentially relevant person with the required skills (as in 4 above) is identified and appraised
• Their particular approach and perspective
• The extent of diversity of topic skills and perspectives required

Some of the science advisers seem to write academic papers together which suggests that personal recommendations may be a factor, though this may also reduce diversity of perspectives and maybe skills too.

5. Structure and functioning

A further issue is the ways in which the committees function and the relationship between them. For example:

• Functioning as a team or group: when they are very large with shifting participation
• Relationships between different actors: such as different perspectives and advisers not necessarily agreeing with policies enacted
• Agenda setting: the extent that discussions are evidence/adviser or policy driven and the freedom to  raise issues and options
• Committee processes for discussion and decision making: including the type and level of detail of decisions, the process by which the decisions are made (such as choosing between similar types of evidence from different research centres), trade-offs between different options and the need (or not) for consensus
• Transparency of reasoning leading to a decision
• Balance of ‘advice’ from committee and its interpretation and development by the CSO.
• Accountability for decisions: degree of and where located (individual, committee, or chairperson)
• Public or private opinions and decisions: the extent that these are shared within or outside of the committee
• Pre-planning: the extent that evidence is prepared as a potential future resource (and where in the system)
• How committees relate and feed into each other: mechanisms and responsibility.
• How ‘advice’ is used and the capacity for the system to absorb and use such information.
• Political dimensions of such use.

6. Relative lack of study of this (except health research systems)

As previously mentioned, the process by which research evidence is used is a topic of social science research that can be applied to all of the issues discussed above. It is in a sense a unique form of social science as it applies to all of the other academic disciplines.

Initially the emphasis of such ‘research on research use’ was rather production or ‘push’ focused’. A rather linear process from research to its use was assumed. Now research use is seen as more of a dynamic two way process where demand or ‘pull’ for research as well as ‘push’ from already existing research (as in Figure 1).

Despite this, ‘research on research use’ is still a relatively undeveloped area of study. ESRC,  for example, invests significant sums on initiatives for increasing the use of research (such as pathways to impact, secondments to government departments, fellows to support government departments’ Areas of Research Interest, What Works Centres). Outisde health, however, there seems to be little investment in the study of the processes and effectiveness of such initiatives or in the study of evidence use more generally.

In terms of global outbreaks of disease, the following topics are highly relevant:

• Structures and processes for consideration of evidence: such as advisor roles, types of expertise, advisory committees, and stages in evidence use ecosystems.
• Mechanisms to enable the use of evidence: Access to evidence; Relationships between decision makers and researchers; Agreement about evidence; Capacity, opportunity and motivation to act.
• Perspectives defining evidence production and use: Selecting questions and evidence criteria.
• Processes influencing decision making: Assumptions, rationality and heuristics; Psychological biases; Integrating evidence driven by different values and priorities within and across different disciplines; Balancing drivers, risks, costs and benefits;  Science and uncertainty; Precautionary measures and adherence to preparedness; Transparency of processes; Communication and  implementation of decisions.

Recommendations

• A more overt clarification of the relationship between the roles and methods of these different systems and structures.
• Analysis of the aim and methods of the current evidence infrastructure.
• An assessment of whether this meets its aims.
• The use of ‘research on research use’ to assess the appropriateness of the current systems and any future changes.
• Development of ‘research on research use’ as a resource to assist the use of evidence in policy making in general and in response to global outbreaks of disease.

Illustrative references

• Gough D (2019). Submission to the House of Commons Liaison Committee on The effectiveness and influence of the select committee system. Fourth Report of Session 2017–19
• Gough, D., Davies, P., Jamtvedt, G. et al. Evidence Synthesis International (ESI): Position Statement. Syst Rev 9, 155 (2020). https://doi.org/10.1186/s13643-020-01415-5
• Gough D, Maidment C, Sharples J (2018). UK What Works Centres: Aims, methods and contexts. London: EPPI-Centre, Social Science Research Unit, University College London.  ISBN: 978-1-911605-03-4
• Hanney, S.R., Kanya, L., Pokhrel, S. et al. How to strengthen a health research system: WHO’s review, whose literature and who is providing leadership?. Health Res Policy Sys 18, 72 (2020).
• Kenny C, Rose, DC, Hobbs A, Tyler C, Blackstock J (2017). The Role of Research in the UK Parliament Volume One. London, UK, Houses of Parliament.
• Langer L, Tripney J, Gough D (2016). The Science of Using Science: Researching the Use of Research Evidence in Decision-Making. Report for the Wellcome Trust. London: EPPI-Centre, Social Science Research Unit, UCL Institute of Education. ISBN: 978-1-907345-88-3
• Parkhurst J (2017). The politics of evidence: from evidence based policy to the good governance of evidence. Routledge – Routledge Studies in Governance and Public Policy. 
• Salajan A, Tsolova S. Ciotti M, Suk JE (2020) To what extent does evidence support decision making during infectious disease outbreaks? A scoping literature review, Evidence & Policy, 1–23

(July 2020)

Page 4 of 4