Written Evidence Submitted by Simon Briscoe
(CLL0096)
I am an independent consultant and a director of a data science company. I have decades of experience in statistics relating to public policy, economics and business. I have worked at the FT (as statistics editor), in investment banking, and as a civil servant in the Treasury and ONS. Full details can be found at simonbriscoe.com. I have had a long-standing interest in data and have published several books on the topic.
Submission:
It is recognised that it is a duty of government to publish data on important issues relating to the state. Data also offers a great opportunity to make better public policy, increase scrutiny and engage the public in important issues.
Yet, it is fair to say that when it comes to that strand of the committees’ report on “modelling and the use of statistics”, the pandemic has revealed terrible weaknesses in delivery. There have been successes but they are outweighed by many failures.
In the first phase of the pandemic (spring 2020), there should have been structures in place to be open with the public and publish the data, models and information that was available. It is quite likely that the public response would have been better if there had been less reliance on secrecy, rumour, selective headlines and dodgy social media. Lessons should have been learnt from the early weeks and months so that improvements could be made for the summer. By then the nation should have been better equipped in terms of new infection and health surveys, good data flowing from the new admin systems (such as track and trace), and a clearer presentation of information (and justification of policy choices).
Having followed the Covid-19 epidemic closely I hope that the committees’ report makes strong recommendations in the following areas:
Some government policy domains have heavy data elements. The understanding of Covid-19, the consequences of the policy and behavioural responses (positive and negative) is very data-heavy so numerical analysis ought to be at the heart of policy development and implementation.
Transparency is the core requirement in an uncertain and rapidly evolving situation if the public is to be asked to make significant sacrifices. All evidence considered by government and its advisers, including for example sage, must be published in a timely fashion. People will excuse a lack of knowledge when much is unknowable but not lies, concealment and deceit.
It is nigh on impossible to know what government-produced data, analysis and announcements exist let alone find it all. There needs to be a portal – a web page of links – which allows official data and analysis to be found. Government commissioned, publicly-funded, work produced by non-government bodies must be listed. The political and professional record of key players such as those in sage should be in the public domain, and confirmation hearings might be held, as is the case for members of other key bodies such as MPC and OBR.
Key NHS data was withdrawn from regular publication, unpublished data was referred to by officials and ministers and some new administrative systems (eg test and trace) failed to produce data of real value. The huge howling gap was the care home sector. Nearly half the excess deaths were in care homes despite under 1% of the population living in such homes. The virtual absence of official data helped to make the sector ‘invisible’ relative to the attention-seeking NHS, leaving residents to die. Had there been data the number of deaths would have been fewer as government would have acted.
Key pieces of information are often missing from research papers and statistics, and the failure to note details on uncertainty in estimates for example devalues the content. This information is especially important if the study is relating to infection routes or treatment of Covid-19 and can directly affect the well-being of individuals. Data publishers need to explain any concepts key to decision-making, such as R, and how the various measures of it relate to each other.
The public sector needs to adopt existing open data principles[1] for all Covid-related data. It is important to allow as much source data and model coding to be published as possible, and make it accessible to all (at appropriate levels of anonymity/security). This applies as much to the complex models as it does to data displayed in a chart or map. This would boost confidence, promote understanding and allow replicability.
The authors and producers of data and reports must give due consideration to a variety of user types. Some will want a plainly written summary while others will want to be able to download and anaylse the code and raw data. Outputs need to range from carefully chosen easy-to-read releases, to data-visualisation, through to APIs for developers.
While some official datasets such as ‘weekly deaths’ have evolved, it is unacceptable that too many key statistics, not only relating to health but also to the economy or public spending, have remined unchanged and thus fail to inform the public (and parliament) about developments. The publication of some data has been stopped.
Repeated statistical research has shown that individuals relate much more positively to data that is relevant to them (‘local’, say, more than national averages which can seem distant). When the buy-in of individuals is needed as a key part of the policy response this data needs to be collected, published and used. The PHE map showed potential but had endless issues such as the format changing and data being hard to download. Publication of personalised risk data would have helped the country return sooner to higher activity rates after the first wave, as in many other countries, where people had not been frightened by media and point-scoring politicians.
Even though the performance of the public sector or publicly-funded front line bodies is key to the epidemic response no up to date data is published setting out activity levels. The publication of some pre-existing core statistics – on, say, NHS performance – has been stopped “due to coronavirus” just when the data was really needed. New timely performance data for the NHS, PHE, LA health officials, test and trace, hospitals and GPs etc were needed and is key to reliable media reporting, public scrutiny, efficient public funding and reassuring the public. Data about staffing, illness, infection, retraining and so forth is not available.
The failures listed above largely result from there being no one in government able or willing to lead, no apparent initiatives from the civil service and too many arm’s length bodies lacking direction. Government has a duty to provide a data infrastructure for its country – if it doesn’t know what’s going on why would anyone trust it and the policies? A National Data Strategy[2] consultation is underway and it must evolve to reflect the weaknesses in the present structure. UKSA looks after only ONS and has no control over other departments. Arm’s length bodies (like UKSA) but also PHE and NHS are free from ministerial control so focus on their own needs not those of the country. The regulator, OSR, is too weak and needs new independent governance.
There have been thousands of reports produced by academics, think tanks and charitable foundations. Yet these are very hard to find – you don’t even know what you are looking for. Universities and charities as employers, and funding agencies, should ensure these reports are accessible via a single web page containing links. Parliament should ensure that links to Covid-related debates and committee reports are readily accessible on such a portal.
Biased, inaccurate and sensationalist reporting surely hampered the UK’s recovery relative to other countries. As poor reporting is often alarmist, it needs to be minimised. Bodies passing on biased or inaccurate reporting should be called out for doing so. Charities, fact checkers and others providing a valuable service need to be supported.
Debate is part of politics and policy making – and life. But the collective failure of local, regional and national governments to work together to discuss policy based on a common evidence base has been damaging. Local and regional data must be consistent subsets of national data. The authorities should present evidence for, and debate, the divergence in their policies. The failure to do so undermines the value of data and reduces policy effectiveness. It has led to unsightly bickering and an appearance of poor governance just when people needed leadership most.
The BBC, as the nation’s dominant publicly-funded broadcaster, has produced some excellent reporting and analysis on Covid-19 but also much in its mainstream news that is poor. The organisation ought to be encouraged to work with experts in and out of government to deliver public information instead of, at least or alongside, the sensationalism in its core news outputs.
The failure of governments and international bodies to compile reasonable, comparable data of even the most basic statistical measures has been a major weakness that needs to be addressed. It limits the scope for international comparison and the ability to learn for other countries’ policies. That said, the UK should produce what comparisons it can to discover what worked.
The government must explain how much money has been spent on data collection and research. Not properly investing in information gathering is a weakness in such situations but so is spending too much money on poor research that will be of little use.
I consider that the cost of most of these changes will not be significant: £10s of millions a year compared to the billions spent on Covid-19. The sense of national purpose and drive for shared data and evidence has been lacking across the political, government and academic fields. There are thousands working in these areas on the public sector pay roll and many must have seen the inadequacies, and perhaps wanted to do something, but were unable to act for some reason. The legacy of the tragic events of 2020 could be a far superior machine for delivering good statistics, data and evidence in future, and better policy.
[1] https://www.gov.uk/government/publications/open-data-white-paper-unleashing-the-potential https://gss.civilservice.gov.uk/wp-content/uploads/2012/12/Open-Data-and-the-GSS.pdf
[2] https://www.gov.uk/government/publications/uk-national-data-strategy/national-data-strategy