Written evidence from RSS (STI0017)
1.1.1 The Royal Statistical Society is a professional society for statisticians and data scientists, with over 11,000 members. Our vision is a world where data is at the heart of understanding and decision-making. Our members include experts in a wide range of statistical and data science disciplines, some of whom have been involved in providing expert evidence as part of statutory inquiries.
1.1.2 Our submission seeks to highlight the crucial need for consideration of data and statistics as part of any inquiry - from the outset as a de facto requirement. This is to ensure that inquiries can be effective, efficient, and include statistical expertise.
1.1.3 Inquiries risk significant resource waste, unnecessary delay and mistaken lines of inquiry if adequate statistical skills are not made available to the inquiry team. The risk will be higher for inquiries where statistical and data issues have not been adequately considered in the terms of reference (ToR) resulting in an over reliance on testimony rather than evidence supported by data. The ongoing Covid-19 inquiry exhibits some of these characteristics.
1.1.4 Our broad recommendation is that:
The role of statistics and data should be considered at the ToR stage of any inquiry, and the core team of any inquiry should include an individual or group with sufficient statistical and data expertise to determine areas that require statistical thinking and recognise when additional expertise is necessary.
1.1.5 To ensure that data and statistics can support inquiries effectively, inquiry teams must understand how evidence and facts are supported by data and as such by the appropriate collection, analyses and interpretation of such data. Furthermore, inquiry teams must be able to interpret confidently the levels of evidence presented to them and be able to access additional expertise where required.
1.1.6 Inquiry teams need to be fully aware of how statistics can support inquiries – particularly in relation to bias and uncertainty. Statisticians can also conduct additional investigations to further understanding. Examples of statistical research commissioned to further understanding include statistical modelling to estimate the number of people who acquired Hepatitis C Virus infection through blood transfusion in the Penrose Inquiry and Infected Blood Inquiry, and record-linkage to determine how the relative risk of dying reduced with time since transfusion in the Infected Blood Inquiry.
1.1.7 Other examples of the use of statistical evidence in inquiries include the calculation of excess deaths for the Bristol Inquiry into children’s heart surgery and anomaly-detection to investigate whether the frequency of Shipman-certified deaths would have been detectable earlier by registrars in the Shipman Inquiry. Anomaly-detection may also be useful in the Post Office Horizon IT Inquiry, to consider whether patterns of alleged fraud in the Post Office’s private prosecutions before and during the Horizon-era were unusual.
1.1.8 An example of the misrepresentation of statistical evidence is the Sally Clark court case. A medical expert witness provided an erroneous calculation that may have influenced the trial and was subsequently misinterpreted by the media (as per the ‘prosecutor’s fallacy’). The RSS has advocated for steps to ensure that statistical evidence is presented appropriately, including the presentation of statistical evidence only by qualified statistical experts, since 2002. The RSS President wrote to the Lord Chancellor about this matter at the time and the RSS has since produced guidance to assist judges, lawyers, forensic scientists and other expert witnesses with the communication and interpretation of statistical evidence in the criminal justice system.
2.1.1 Below, we set out in more detail our views and recommendations with respect to three key aspects of an inquiry: 1) evidence collection, 2) interpretation of evidence (by inquiry panel), and 3) independent expertise.
2.2.1 A key aim of an inquiry is to collect evidence effectively. The evidence supplied to an inquiry is often supported by, or based on, data. However, there is not always sufficient consideration regarding how the data was collected, whether all relevant data has been collected, and whether the analysis and presentation has adequately addressed the potential for bias.
2.2.2 Consideration of statistical methods, including random sampling, to avoid ascertainment bias, to support generalisability of findings and to ensure cost-efficiency, has a key role in data collection. Such statistical methods and data science can be critical to understanding the patterns and root causes of observed anomalies.
2.2.3 The RSS has previously written to the chair of the Thirwell (Lucy Letby) Inquiry to request that the ToR include the appropriate use of statistical evidence. We were subsequently pleased to see that the use of hospital data is now included in the ToR and believe that at their outset, all inquiries should consider the appropriate use of data and statistics.
2.2.4 We recommend that the ToR of any inquiry should explicitly consider how data will be collected, collated, and presented to provide evidence to support the inquiry. Consideration should be given at an early stage as to whether key data is missing and whether further data collection is necessary.
2.3.1 The interpretation of data as evidence requires expertise based on sound statistical principles and thinking. The default position should be that the core inquiry team includes a statistician (or Statistical Expert Group, as in the Infected Blood Inquiry) who is able to interpret evidence, assess where statistical work should be commissioned to inform the inquiry, and recognise where external deep expertise is necessary and should be sought to advise the panel.
2.3.2 There are examples of RSS members being called to explain simple statistical concepts regarding data that had been submitted to an inquiry. While the RSS is pleased to connect inquiries with expertise, inquiries are likely to be more efficient if there is a basic level of statistical understanding on the inquiry team itself.
2.3.3 We acknowledge that statisticians must be cognisant that cases around human beings are not simply numbers, and that statistical evidence must be presented with sensitivity. In addition, statisticians must be aware of the potential media and public interest and how evidence may be construed – taking care to limit any opportunities for misinterpretation.
2.4.1 The core inquiry team needs to know when and how to seek external statistical expertise – whether this be expert statistical witnesses or the commissioning of specific statistical analyses. We encourage the inclusion of relevant expertise.
2.4.2 Statistics and data are ubiquitous. Statisticians are accustomed to collaborating with professionals in other fields and we note that the most effective use of statistical expertise could be in conjunction with non-statistical expertise (e.g. experts in surgery or haematology in inquiries around health).
2.4.3 In instances where external experts flag areas that require further data-led investigation, we would expect that such concerns are recognised by an inquiry and that their suggestions are explicitly considered.
2.4.4 The RSS has a rich network of statistical expertise and would be pleased to support this process.
2.4.5 As a final and separate point, in line with our thinking on ‘public statistics’ (statistics to meet public need) and vision that statistics and data should be used to inform understanding and public debate, we would advocate for any data and statistics used as part of statutory inquiries to be accessibly-explained, clear and readily-available to help inform public understanding.
6