Written evidence from Full Fact (DTA 21)

Public Administration and Constitutional Affairs Committee

Data Transparency and Accountability: Covid 19

Summary

●       Full Fact works to reduce the harm that bad information does, and advocates for changes that will increase the availability and accessibility of good information.

Data and information

●       The need for robust and timely data has been made clear by the coronavirus pandemic, which has exposed the problems caused by fragmented or partial data sources. Done well, the collection of good data can improve policy responses, public services, provide cost savings and benefits, and help build and maintain public trust.

●       To do this, the government must make long-term investments in data infrastructure, data quality, data skills and horizon scanning to ensure it is better able to anticipate, react and recover from crisis events.

Communication

●       Full Fact has published hundreds of articles about the pandemic, and followed up on dozens to ask claimants to correct the record. We have had thousands of requests from the public, indicating confusion over lockdown rules and medical concerns.

●       We are particularly concerned about the quality of responses to questions we posed to the government. These were too often slow, unclear or inaccurate; we were told contradictory things and faced unwillingness to engage with questions of accuracy.

●       While recognising the pressures the government is under, standards of accuracy must be higher. It is essential that departments actively encourage a culture that emphasises the importance of transparency and evidence, and the need for mistakes to be corrected quickly and openly.

●       We are already seeing misleading claims spreading about the coronavirus vaccine. It is vital that the government acts without hesitation to provide clear details of the process of developing and testing vaccines.

Trust and transparency

●       A pandemic does not reduce the need for scrutiny of decisions; arguably it increases it, as more draconian measures may be sped through in the name of tackling the outbreak. This relies on transparency and good communication from those in power, and this openness is essential for public trust.

●       However, the public does not owe decision makers their trust - it must be earned. Efforts to improve data use will be hindered if the public lacks confidence in those setting the policies or delivering the services. The legacy of past controversies have damaged public trust, and this list has been added to during the pandemic. Debate about the use of data must be had in, and with, the public.

About Full Fact

1. Full Fact fights bad information. We’re a team of independent fact checkers, technologists, researchers, and policy specialists who find, expose and counter the harm it does.
2. Bad information damages public debate, risks public health, and erodes public trust. So we tackle it in four ways. We check claims made by politicians, public institutions, in the media and online and ask people to correct the record where possible to reduce the spread of specific claims. We campaign for systems changes to help make bad information rarer and less harmful, and we advocate for higher standards.
3. We fact check claims from public figures, including politicians, that we see in the traditional media, and on social media. Full Fact is a UK partner in Facebook’s Third-Party Fact-Checking programme, which gives us access to a queue of posts being shared in the UK that have been flagged as potentially false by Facebook and Instagram or its users, and which we can fact check and attach ratings to.[1] We recently launched a three month pilot partnership with WhatsApp that allows the public to send in claims they have seen being shared. Our fact checks have also been integrated into Google search since 2017, and will be a source for the new fact-check boxes on YouTube.[2]
4. Full Fact is a registered charity. We're funded by individual donations, charitable trusts, and by other funders. We receive funding from both Facebook and Google. Details of our funding can be found on our website.[3]

Good information is essential to decision making

5. Good information allows us to answer the most pressing of society’s questions; by quantifying a problem, the government is in a better position to tackle it. Relying on poorer quality information, or simply not having it, risks costly delays in action.
6. The public benefits not just from improved public services they can place more confidence in, but also from having access to accurate information when making decisions in their own lives.

Decision makers need the right information at the right time

7. The coronavirus pandemic has shown that robust and timely data is crucial to support decision making. It has helped to coordinate support for the people that needed it.[4] Statistics producers have acted quickly to stand up new surveys; adopted new tools, methods and data sources; and adapted existing methods to continue collecting vital information about our society.
8. The statistical system has historically been too slow to take up new ideas, and we among many others have called for more ambition and agility. Where the pandemic spurred faster action and novel approaches, we urge producers to maintain this momentum. This must consider data governance, security and privacy concerns, along with caution where necessary, for instance in balancing data quality with speed.
9. The pandemic has also exposed fragmented or partial data sources - like the black hole in the social care system[5] - and brought reports of problems with data sharing within and between branches of government.[6] Inconsistencies make it hard to use data effectively, slow down the work of analysts who are trying to combine different datasets, and can cause confusion among the public.
10. But the government must not rely solely on data already collected - which may be based on past priorities - but look to the future. We have urged the government to establish a horizon-scanning function for statistics and data, led by the UK Statistics Authority. This should assess what the major decisions of the next five years will be; ask whether we have the data, statistics and analysis we need; and whether they are being communicated effectively. Where gaps are identified, work should be done to gather the relevant information, from a variety of sources if necessary.
11. The UKSA’s five-year strategy, published in July, has a promising pledge to be “ambitious”. This commits it to set out to answer critical research questions and inform the decisions that people and organisations take, and to “anticipating the data, insights and understanding the UK needs”.[7] We await further detail on how it will meet this commitment and answer crucial questions about the work, such as over what time period it will be looking ahead, how it intends to engage with decision-makers, and how it will communicate its work plan to stakeholders.

Taking advantage of data requires long-term thinking

12. Governments have for too long struggled to take best advantage of data, especially when dealing with inadequate data infrastructure, out-of-date technologies or systems where interoperability is a challenge. Piecemeal investment makes it difficult for departments to plan for the future and ad hoc projects risk being deprioritised or defunded when budgets have to be reduced. Even major efforts can run aground without sufficient high-level or sustained support; there have been numerous previous attempts at government digital or data transformations.
13. The UK needs strong, long-term investment in systems, with data that is fit for purpose, recorded in standardised formats on modern, future-proof systems and held in a condition that means it is findable, accessible, interoperable and reusable.
14. At the same time, the government must invest in skills to allow proper advantage to be taken of improvements in data and information accessibility and analytics. This will drive up demand for information, as civil servants gain a greater understanding of how data could benefit their work. Improved skills across the civil service will also benefit those communicating with other stakeholders and the public.
15. In addition to increasing spending commitments in infrastructure, skills and tools, a renewed focus on data governance, transparency and accountability is essential. Without this it will be impossible to earn the public’s trust, which is essential for a properly functioning democracy.
16. Done well, there is an opportunity to collect the right data that feeds into a richer set of evidence to inform and evaluate policy decisions over both the short and long-term. This should improve policy responses over time, improve public services, provide cost savings and benefits, and help build and maintain public trust.
17. Beyond the information that the government produces, uses and communicates, there is a wider need for better information sharing across sectors. This could be a foundation for maintaining the supply of reliable information in crises. Now is the opportunity to take a carefully considered and cohesive approach to how we want to tackle these problems in the future. To achieve this, Full Fact is over the next few months convening experts to create a new model for approaching information crises.[8]

Government communication during the pandemic

18. The government needs effective communication to reassure concerned citizens, help ensure that official guidance is followed and allow other stakeholders to make their own decisions. In keeping the public informed, it must explain complicated problems, deal with uncertainty and guard against providing so much information it is rendered useless.
19. Good communication is also essential for transparency and accountability. A pandemic does not reduce the need for scrutiny of government decisions; arguably it increases it, as more draconian measures may be sped through in the name of tackling the outbreak.

Our experiences fact checking the pandemic

20. As fact checkers, Full Fact has direct contact with government departments through press offices. We found responses were too often slow, unclear or inaccurate; we were told contradictory things; and were provided with information ‘on background’, which means we are unable to attribute it to the person or department that provided it. We have found that statistical requests are not always properly understood - an issue that has become more apparent during the pandemic, as we deal with more complex issues relating to new and evolving datasets.
21. More worryingly we have even faced an unwillingness to engage with questions of accuracy. For example, in June, the Prime Minister claimed that all tests at testing centres and mobile testing units at the time were turned around within 24 hours. We asked the Department of Health and Social Care about this figure at the time, and were told that it was correct. However, data published at a later date showed that the number of tests turned around in that time was much smaller. When we put this to DHSC, it did not respond directly. We brought our concerns about both the inaccurate figures and the poor communication in a letter to the OSR[9], and were told that the matter had been raised with the department.[10] The Prime Minister has failed to correct the official record on this and a number of other inaccuracies.
22. We recognise that there are significant pressures on the government at this time, and that mistakes will be made when providing or presenting data. But we believe much more can be done to improve the presentation and communication of data, and there must be a willingness to correct inaccurate claims once they are identified, no matter who said them.
23. It is also essential that all departments actively encourage a culture that emphasises the importance of transparency and evidence to all staff, to ensure that information provided by ministers and departments is accurate. This should include a greater emphasis on the important role of statisticians in producing public-facing documents; and the use of a fact checking and sourcing template document to ensure work is accurate.
24. Where questions are particularly complex, we urge departments to make analysts directly available to the media. There would be benefit in having named analysts on government-produced information, in a similar way to the named statisticians that on releases from the Office for National Statistics.
25. Our Full Fact Report 2020 on the causes and consequences of bad information goes into more detail on our concerns around accuracy, transparency and accountability relating to a number of actors, including the government.[11]

Public understanding during the pandemic

26. The government could have done more to help the public understand the issues they were facing, whether about specific rules or the broader situation and how it applied to them. The early narrative about “following the science” risked confusion over the nature of the scientific method and masked the fact that policy decisions are informed by advice but ultimately taken by ministers.
27. At Full Fact, we spend a great deal of time thinking about the best way to explain and communicate uncertainty. We must carefully balance being explicit about uncertainty and nuance where they exist, while also being clear where we think the evidence points in one direction. This is discussed in more detail in our research briefing, How to Communicate Uncertainty.[12]
28. Throughout the crisis, we have run an ‘Ask Full Fact’ service where the public can put their questions to us.[13] An analysis of these questions gives an idea of the issues that were causing confusion during the pandemic. To date, we have had more than 3,000 requests and while we cannot answer them all, we read them all and group them into themes to make sense of what matters to our audience. The types of questions we received suggested that our readers - who we do not claim to be a representative sample of the UK population - wanted to stick to the rules, but couldn’t easily apply them to their own situation. More than a third (38%) have asked us about rules, while 13% had medical queries and 11% questions about transmission.
29. It is perhaps unsurprising the public had questions, as we saw instances where different ministers and departments provided conflicting information. For instance, the health secretary made a mistake in who had been asked to shield[14]; the foreign secretary Dominic Raab incorrectly stated that government guidance meant people could meet both of their parents at the same time[15], and the prime minister gave the wrong advice about grandparents bubbling with a couple and their grandchildren.[16]
30. There were also instances where ministers seemed to choose certain numbers in order to paint a more positive picture of the situation - for instance when the prime minister overstated the number of schools with returning students,[17] or when the health secretary used a confusing metric about the proportion of tests turned around in “24 hours” that actually included tests that were returned the next day.[18]
31. We are not naive about the nature of politics, and acknowledge fully that mistakes can happen, but accuracy is essential in crises. Failing to ensure this means leaving citizens confused over what they can and can’t do, which has the potential to impact how they go about their lives.
32. Looking to the future, it is crucial that the government seeks to predict and prevent potential misunderstandings or misconceptions, especially when this might cause or stoke public fear. This is particularly true for misinformation that swirls online and then spreads into the mainstream news, and it is essential that the government gets out ahead of this.
33. At the start of the pandemic, we saw one example of this, where conspiracy theories about 5G that had been circulating online for years started being linked to coronavirus. These spread into traditional media, and appeared to take hold in the public, with mobile phone masts being vandalised and network engineers confronted by members of the public.
34. In recent weeks, we have seen a number of claims about vaccines catch the public’s attention.[19] There is a need for the government to tackle the issue of vaccine misinformation before the arrival of a viable vaccine. Our recent research briefing on conspiracy theories shows us that the more effective means of countering vaccine misinformation is through preventative measures, such as showing people debunks of anti-vaccination claims before the original conspiracies.[20]
35. It is vital that the government acts without hesitation, providing clear details of the process of developing and testing vaccines for Covid-19 and what is known about how safe they are. The country’s ability to recover from the pandemic rests at least in part on vaccine uptake.
36. Full Fact also believes strongly in providing everyone with the tools and skills they need to find good information so they can make their own decisions. We have invested in creating resources that seek to improve audiences’ ability to think more critically about the information they come across. The UK government’s forthcoming media literacy strategy should encourage coordination and evaluation across organisations, as well as provide sufficient investment in education across all ages.

Transparency and public trust

37. The pandemic has demonstrated just how powerful proper collection, access to and sharing of data is - but these efforts will be hindered if the public lacks confidence in those setting the policies or delivering the services. There have been examples where poorly communicated data or confusing messaging has had a damaging impact on public opinion and trust.[21] Misleading or manipulative use of statistics can similarly damage debate and undermine public trust.
38. Never has it been more essential to expose misuse of statistics and challenge those who seek to mislead. Full Fact shares this aim with the Office for Statistics Regulation, which works to help to ensure statistics are trustworthy, transparent and well communicated. We welcome the fact the Committee has closely monitored the OSR’s work during the pandemic. It is crucial that this continues, and important that the Committee is willing to act in support of the regulator where necessary and appropriate.
39. The legacy of past controversies have impacted public trust, and this list has been added to during the pandemic. The use of algorithms to award exam results is a stark example of the need for proper transparency and communication with the public. The error of using an outdated version of Excel to save Test and Trace data led to some 16,000 records going temporarily missing. Such incidents do little to build the public’s trust in the government’s use of data; a failure to be transparent exacerbates the problem.
40. In contrast, surveys have found that people are more willing to accept the use of their data if they believe it is for the public good[22], and are more willing to trust organisations when they are transparent on how they use data.[23] Public awareness will be crucial in gaining the public’s trust, and there must be a focus on  preventing the introduction of any perverse incentives that might encourage less transparency.[24]
41. As Full Fact and others said in an open letter last year:
    
    “Debate and discussion about the appropriate extent of using citizens' data within government needs to be had in public, with the public. Great public benefit can come from more joined-up use of data in government and between government and other sectors. But this will only be possible, sustainable, secure and ethical with appropriate safeguards, transparency, mitigation of risks and public support.”[25]

Media coverage during the pandemic

42. Since the outbreak of Covid-19, Full Fact has published hundreds of fact checks and articles providing the context and evidence behind claims about the virus, the lockdown, and the government’s response to the crisis.[26] As well as checking claims made by public figures, and monitoring social media, we also look for claims in print and online newspapers, and in broadcast media including news programmes on TV and radio. This gives us a unique view of the pandemic and coverage by the media.
43. When we see claims that are incorrect or misleading, we follow up with the person who made them. So far in 2020, we have requested around 60 corrections to media outlets about the pandemic, of which just over half have been resolved.
44. Claims in the media tended to align with public debate and information communicated by the government, with an initial focus on the origins of the outbreak and symptoms, followed by coverage of lockdown and then local lockdowns, alongside the continued impact on day-to-day life and the government’s response. Throughout there has been a strong focus on data, with regular updates on death tolls and government targets, and scientific research.
45. Clear communication of uncertainty and the scientific process were essential for the media as much as the government. Over time, we have seen some improvements, for instance in more clearly expressing uncertainty or limitations in data by providing the reproduction, or R, number as a range and not a single number, or ensuring that it is clear when weekend time lags are affecting mortality figures.
46. But we also saw research papers - including those published on preprint servers that have not yet been peer reviewed - that would usually go unnoticed by the media or general public made headlines. Trying to unpick complex materials can and did lead to significant errors in some newspapers. This included an article in the Express - now corrected - that misinterpreted the results of a genetic study as saying Covid-19 had been “genetically engineered for the ‘efficient spreading in the human population’”.[27]
47. We also saw repeated misinterpretations of an ONS release that compares “influenza and pneumonia” (a grouping that can include people who have either or both illnesses) with Covid-19. Various media outlets ran stories with headlines like “Flu killing six times more people than coronavirus” or “Flu killed 10 times more Brits than coronavirus last week for 14th week in a row, new stats reveal”. However, the ONS data looks only at the number of death certificates that mention these illnesses, not what the underlying cause of death was. This is important, because when Covid-19 is mentioned on someone’s death certificate it is much more likely to be the underlying cause of someone’s death (in 93% of cases), than when flu or pneumonia is (in 28% of cases).[28] In this case, misinterpreting the data could give the public the false impression that Covid-19 was less of a risk than it was in reality.
48. This and our broader experience fact checking the media shows how important critical thinking is to journalism, especially when covering complex studies or datasets. However, we also acknowledge the pressures that the media is facing, especially at the moment, and recognise that as face checkers we have the luxury of time. We know only too well how complicated it can be to scrutinise this information, because we have sometimes spent days fact checking them. But being clear about exactly what is and isn’t backed up with evidence should be the minimum.
49. There are also occasions when the way information has been presented to the media has been lacking in these standards of accuracy and clarity. It is incumbent on organisations like universities to ensure that press releases properly present a study’s caveats and limitations. Similarly, producers of data and government officials must consider how data may be used by others and seek to protect against misinterpretation.

October 2020