Written evidence from Dr Subhajit Basu[1] (TEB02)

Public Administration and Constitutional Affairs Select Committee

Transforming the UK’s Evidence Base inquiry

I am a socio-legal scholar specialising in the intersection of Law and Technology. My research encompasses various facets of technology regulation, making notable contributions to the critical policy discourse surrounding AI, Big Data, health data, autonomous systems and more. My work examines crucial issues such as data protection, privacy and accountability.

1. The changing data landscape

1.1               Is the age of the survey, and the decennial census, over? 

The decennial census is a monumental exercise in democracy. However, what appears to be a straightforward task is fraught with complexities and vulnerabilities. The digital age has ushered in a transformation in how information is gathered, analysed, and leveraged. This shift impacts many domains, including traditional data collection methods like surveys and Censuses. With the advent of big data, AI, and real-time analytics, these conventional methods may be outdated[2]. A decennial Census is often too slow to capture the rapid changes in population dynamics.

The decennial census has a storied history, reflecting both the noble ideals of democratic representation and the darker undercurrents of human nature. In some instances, the census has been used to reinforce racial biases. In the US, for example, the infamous "three-fifths compromise" counted enslaved individuals as only three-fifths of a person for representation purposes[3]. Contemporary challenges include undercounting minorities, which can diminish their political representation and access to resources. Sometimes, ignorance plays a role in skewing the census, whether wilful or not. Misunderstanding the purpose of the census, fears of government intrusion, or lack of awareness about the importance of participation can lead to lower response rates and inaccurate data[4]. Today, the process faces new and evolving threats:

• Disinformation campaigns: Spreading false information can discourage participation or create confusion, leading to an inaccurate count.
• Legal and political battles: Contentious debates over including specific questions, such as citizenship status, can turn the census into a political battleground with implications for marginalised communities.

The decennial census has been a crucial aspect of our democratic society but has flaws[5]. Although we cannot deny the importance of an accurate count, questions arise about the efficiency and allocation of resources involved. Could we spend the money more effectively? Are there better alternatives or improvements that are more cost-effective? One suggestion is to supplement the decennial census with more frequent smaller surveys that provide more timely and accurate data. The census process could become more efficient, accurate, and cost-effective using advanced technology, AI, and big data analytics. We should see new technologies as complementary, rather than competitive, with traditional methods. Big data and AI can provide broad, rapid insights, while traditional surveys offer depth and specificity. With their established protocols, traditional methods can validate and contextualise findings from big data and AI. However, we must remember that not everyone has equal access to digital platforms, and relying solely on new technologies might exclude certain populations. Traditional survey methods can be tailored to different cultural contexts, which might be challenging for AI and big data to replicate.

We must adapt the census to reflect these changes as our society progresses. The reimagining of the census goes beyond cost-saving measures or adopting new technologies; it recognises its significance for our nation. We must ensure that the census accurately and impartially represents our diverse and continuously evolving society. By incorporating innovations while closely monitoring ethical and precise practices, we can create a census suitable for our future needs. This approach will uphold the democratic purpose of the census while adapting to modern values. In today's data-driven world, it is essential to balance traditional methods and digital advancements to maintain the richness and humanity of the census.

1.2 What new sources of data are available to government statisticians and analysts?

In recent years, government statisticians and analysts have found themselves with access to an unprecedented array of new data sources. These emerging sources, stemming from technological innovations and shifts in social behaviour, provide novel opportunities to understand and engage with various aspects of society. In the following discussion, I will explore some of these "new" sources, outlining their nature, potential applications, and the challenges they present.

Social media have revolutionised how people communicate, share their views, and interact with the world. This digital interaction is a valuable resource for government statisticians and analysts, who can use it differently. Users often share their thoughts, experiences, and opinions on social platforms. These "public posts" can be a rich source of insights into public sentiment on various topics. By analysing the likes and reactions to posts, governments can gauge the popularity or acceptance of particular ideas, policies, or public figures. Trending topics and hashtags reflect the real-time concerns and interests of the public, offering a pulse on the nation's mood and preoccupations. The comment sections of social media posts can provide deeper insights into public opinions, including diverse perspectives, debates, and detailed feedback.

Sensor Data and Internet of Things (IoT) devices can rapidly transform how governments collect and analyse data. The information generated from these technologies provides invaluable insights that can drive better decision-making and more efficient public services. It is possible to collect data from connected devices (it can be done within the existing legal framework) such as:

• Traffic Cameras: Capture real-time traffic conditions, aiding urban planning and transportation policy.
• Weather Stations: Offer immediate climate and weather data essential for disaster management and agricultural planning.
• Smart Meters: Track energy consumption at the granular level, enabling more efficient energy distribution and conservation policies (but this will raise questions about privacy)

Web scraping and search data analysis offer innovative ways for government statisticians and analysts to gather information. By aggregating publicly available data from websites or scrutinising search engine queries, they can attain insights that inform policymaking, public service delivery, and societal understanding.

Satellite and remote sensing data open unprecedented opportunities. This technology is transforming how we gather, analyse, and interpret information on a scale never imagined. From managing natural resources to enhancing national security, the applications are diverse and transformative. These technologies provide a comprehensive, real-time picture that informs policymaking, development initiatives, and environmental stewardship.

Healthcare data, particularly from Electronic Health Records, wearable devices, and medical registries, can become indispensable for government statisticians and analysts. From shaping public health policies to fostering medical innovation, the applications are profound and far-reaching. These data-driven insights offer the potential for more responsive, patient-centred, and efficient healthcare systems (but it also comes with significant privacy concerns).

Public-private partnerships, crowdsourced data, and biometric and security data represent a multifaceted approach to information gathering and utilisation for the government. Together, they offer a rich tapestry of insights that can drive research, innovation, public engagement, and security enhancement.

1.3 What are the strengths and weaknesses of new sources of data?

Understanding the strengths and challenges of various data sources is crucial for optimal utilisation. With advancing technology, these sources will become increasingly significant and may alter our perception and interaction with the world. The new data era is here, and understanding its dynamics is no longer optional—it is essential. The new data era has dawned, making the understanding of its intricacies not just beneficial but necessary. As previously detailed in question 1.2, I have explored some innovative data sources now accessible to government statisticians and analysts. In the following discussion, I will focus on these particular "new" data sources, shedding light on the unique strengths and weaknesses each presents.

1. Social Media

As an increasingly important element of our society, social media offers immediate insights into public opinion and trends due to its real-time nature. Government agencies have access to a diverse and large population sample, often making data collection more cost-effective than traditional survey methods. However, these advantages come with weaknesses. The quality and reliability of posts and comments can often be subjective and unverified. Privacy concerns emerge as there are ethical considerations in collecting and using personal data. Lastly, social media data may suffer from sample bias as it may only represent some of the population, especially those not using social media platforms.

2. Sensor Data and Internet of Things (IoT)

The advent of IoT technology provides precise measurements of various parameters, enables real-time tracking and assessment, and paves the way for integration with automated systems for increased efficiency. Despite these advancements, the need for significant investment in sensors and networking infrastructure, vulnerability to cyber-attacks and unauthorised access, and the need for standards to ensure interoperability and consistency present vital challenges that must be addressed.

3. Web Scraping and Search Data

Web scraping and search data provide vast access to publicly available data. This makes it effective in identifying trends and patterns in online behaviour and can be a powerful tool for monitoring competitors or global developments. However, potential legal and ethical issues could arise, including breaching terms of service or intellectual property rights. The inconsistency and potential inaccuracy in scraped data are a concern, as well as the dependence on website structures that are prone to changes, which can break scraping tools.

4. Satellite and Remote Sensing Data

Satellite and remote sensing data provide the ability to monitor global phenomena like climate change, offer objective data not subject to human bias, and collect long-term data that enables trend analysis over time. Despite these strengths, high costs are associated with launching and maintaining satellites, and specialised skills are required to interpret and analyse the collected data. Moreover, the quality of the data can be affected by atmospheric conditions.

5. Healthcare Data

Digital health records, wearable devices, and medical registries enable tailored medical interventions, support disease tracking and public health decision-making, and facilitate coordinated care across providers. Nevertheless, the sensitivity of health information necessitates robust protection mechanisms, leading to privacy and security risks. Challenges in integrating different systems and formats - interoperability - can also be an issue, as well as the potential for errors in medical records, which can have serious consequences.

6. Public-Private Partnerships, Crowdsourced Data, Biometric & Security Data

Public-private partnerships can leverage diverse expertise and resources, enabling collaborative innovation. Crowdsourcing involves citizens directly in governance and community building, fostering a sense of public engagement. Biometric and security data enhance security and offer robust identity verification. Despite these benefits, potential conflicts may arise in aligning interests in public-private collaborations. Data quality in crowdsourcing poses verification and validation challenges. Additionally, using biometric data presents ethical dilemmas as the balance between security and privacy rights is difficult to maintain.

In the era of big data, various modern data sources present unique strengths and weaknesses. Government statisticians and analysts must understand and manage these to make informed, effective, and responsible use of these diverse data streams.

2. Protecting privacy and acting ethically

2.1   Who seeks to protect the privacy of UK citizens in the production of statistics and analysis? How?

The UK government is facing the difficult task of utilising the advantages of data monitoring while also upholding ethical and legal responsibilities to safeguard individual privacy rights. This delicate balance involves analysing statistics and patterns, considering people, their experiences, and their liberties.[6]

In the UK, data-driven insights play a critical role in government decision-making, and protecting privacy has taken immense significance[7]. The demand for real-time data and in-depth analysis is coupled with a commitment to uphold the individual's right to confidentiality. It is a dual responsibility that requires careful regulation and a transparent and nuanced approach to data collection and analysis.

Ensuring privacy and security in this context goes beyond mere compliance with laws. It is about implementing robust measures to safeguard shared information, understanding and navigating the intricacies of data ownership and control, and considering the societal implications of data usage. How do we reconcile the collective benefits of information with individual citizens' fundamental rights and dignities? It is not merely a technical challenge but a profound societal question that demands a thoughtful and considered response.

In the UK, protecting privacy in the production of statistics and analysis involves several organisations and legal frameworks, each playing a unique role in preserving the integrity of personal information. These include:

• Office for National Statistics (ONS): Implementing strict confidentiality rules and security measures.
• Information Commissioner's Office (ICO): Enforcing regulations like the GDPR to ensure lawful data processing.
• UK Statistics Authority: Overseeing the production and dissemination of official statistics in compliance with ethical guidelines.
• Government Departments and Public Bodies: Adhering to data protection legislation to ensure responsible data collection, storage, and use.
• Technological and Security Measures: Utilising cutting-edge technologies to encrypt and protect data.

These diverse measures reflect a multifaceted approach to privacy protection, embodying the UK's commitment to balance the utility of widespread data monitoring with the ethical imperatives that underpin a democratic society.

The UK's Office for National Statistics (ONS) holds a pivotal role in safeguarding privacy, particularly concerning the collection and analysis of statistical data. Its responsibilities are guided by principles and legal frameworks that aim to protect personal information, ensuring that individuals' privacy is upheld even as valuable statistical insights are drawn from the data.

Adhering to Confidentiality Rules in Line with the Statistics and Registration Service Act 2007: The ONS operates under stringent confidentiality rules defined in the Statistics and Registration Service Act 2007. This legislation mandates that individual data collected for statistical purposes be treated with confidentiality. It places legal obligations on the ONS to ensure that information that could identify individuals is not disclosed inappropriately[8].

Anonymising Data: One of the fundamental ways the ONS ensures privacy is through anonymising data. This means the resulting statistical analyses can be used without revealing information about specific individuals. Data aggregation, masking, and pseudonyms may be employed to anonymise data.

Implementing Robust Security Measures: Beyond the legal framework and data processing techniques, the ONS also implements stringent security measures to prevent unauthorised access to personal data. These measures may include encryption, secure access controls, and continuous monitoring of systems to detect and avoid any potential breaches. The objective is to create a robust defence against cyber threats and unauthorised access that might compromise individuals' privacy.

Information Commissioner's Office (ICO) is central to the UK's data protection landscape. As the independent regulatory body enforcing data protection laws, it has several vital functions that align with safeguarding personal information. Here is a closer look at those functions:

Enforcing the Data Protection Act 2018: The ICO is tasked with ensuring compliance with the European Union's GDPR (which continues to apply in the UK following Brexit under UK GDPR) and the Data Protection Act 2018. These legal frameworks establish the principles and guidelines for how organisations should handle personal data, including consent, purpose limitation, data minimisation, and individuals' rights over their data. The ICO ensures these rules are followed, guiding organisations and taking legal action where necessary.

Ensuring lawful, transparent processing of personal data: One of the key principles underpinning data protection laws is that personal data must be processed lawfully and transparently. This means that organisations must have legitimate grounds for processing personal data and must do so in a way that is clear and understandable to the individuals whose data is being processed. The ICO's role here includes monitoring and investigating practices to ensure that organisations are transparent about their data processing activities and comply with legal requirements such as obtaining proper consent.

Taking enforcement actions against organisations violating privacy laws: If an organisation is found to have violated data protection laws, the ICO has the authority to take enforcement action. This can include issuing warnings, imposing fines, or in severe cases, pursuing criminal prosecutions. The aim is to hold organisations accountable for failures in data protection and to provide a deterrent against non-compliance. Enforcement actions are tailored to the nature and severity of the violation, ensuring a proportionate response.

UK Statistics Authority plays a key role in the integrity and privacy of statistical data within the United Kingdom. It is an independent body that reports directly to Parliament and oversees the Office for National Statistics (ONS). The authority's functions related to data protection and privacy are guided by several principles and guidelines, as outlined below:

Promoting confidentiality and Data Protection principles: The UK Statistics Authority follows the Code of Practice for Statistics. This code lays out a set of principles and practices that all UK government statisticians must adhere to. Among these principles are confidentiality and data protection guidelines, emphasising the importance of protecting personal information and ensuring it is used only for legitimate statistical purposes. The authority actively promotes these principles and guides best practices to ensure that data protection laws, including the GDPR, are observed.

The Code of Practice for Official Statistics plays a significant role in safeguarding privacy, providing specific principles and practice statements concerning confidentiality. Essentially, it mandates that private information about individuals or entities collected in producing official statistics must remain confidential and be used solely for statistical purposes. The practices outlined in the code necessitate striking a careful balance. On the one hand, they must ensure that arrangements for confidentiality protection are robust enough to shield the privacy of individual information. On the other hand, they should not be so restrictive that they unduly limit the practical utility of official statistics.

Furthermore, the code explicitly requires that a written confidentiality protection agreement be in place whenever confidential statistical records are shared for statistical analysis. Additionally, an operational record for each transmission must be maintained. These stipulations underline the meticulous attention to detail and accountability that must be followed, ensuring privacy and utility in handling statistical data.

Ensuring personal data is handled with integrity and the privacy of individuals is respected: Beyond mere compliance with legal requirements, the authority sets a standard for ethical conduct in the handling of personal data. It actively monitors and reviews how data is collected, stored, and processed to ensure that individuals' privacy is respected at all stages of the statistical process. This includes ensuring that personal identifiers are removed or anonymised wherever possible and that strict access controls are in place to prevent unauthorised access to sensitive information. The authority's oversight helps build public trust in government statistics, reinforcing that personal information will be used responsibly and carefully.

Government Departments and Public Bodies in the UK must adhere to data protection legislation. They must implement specific policies, guidelines, and oversight mechanisms to ensure that data is collected, stored, and used in a manner that safeguards individuals' privacy. Additionally, technological solutions are used to enhance data security, including employing encryption and secure access controls. Regular audits and assessments are conducted to ensure compliance with security standards and privacy laws.

The UK has established a robust framework for protecting privacy in statistical production and analysis. By employing a combination of legal, ethical, technological, and organisational measures, the nation seeks to balance the need for accurate and meaningful statistics with the imperative to protect the privacy and dignity of its citizens. This multifaceted approach not only upholds the legal rights of individuals but also strengthens the integrity of the data, promoting public trust in government activities and decision-making processes. It sets a precedent for other countries, demonstrating that rigorous data analysis can coexist with stringent privacy protection.

2.2   What does it mean to use data ethically in the context of statistics and analysis?

Using data ethically in the context of statistics and analysis refers to a comprehensive approach beyond mere legal compliance to encompass a broader consideration of moral principles and societal values. It recognises that collecting, handling, and using data can significantly impact individuals and communities. The UK Statistics Authority has drafted a range of ethical principles, supplemented by an ethics self-assessment instrument. Essentially, these principles strive to secure the public benefit derived from research and statistics, safeguard the privacy of data, recognise and manage the potential risks and limitations of innovative research methods and technologies, uphold legal standards, consider the societal acceptance of the project, and ensure openness in the acquisition, utilisation, and dissemination of data.

Ethical guidelines in statistics and data analysis serve several important functions. Primarily, they encourage a sense of accountability among data practitioners, which is achieved through clear communication of the expectations and standards for their work. In this context, accountability refers to the responsibility of individuals and organisations to conduct their data collection, analysis, interpretation, and reporting in a way that respects certain ethical principles. These principles include fairness, transparency, respect for participant rights, and rigorous adherence to scientific and methodological standards. The aim is to prevent data misuse or misinterpretation, avoid bias or inaccuracy, and ensure respect for the privacy and autonomy of those from whom data is collected.

Those who rely on statistical practices - including policy-makers, businesses, researchers, and the general public - are thus informed of the standards they should expect. They can be assured that the data is collected and used responsibly and ethically, promoting trust in the data and the conclusions drawn from it—society as a whole benefits from these ethical guidelines, as they support informed and accurate judgments. Decisions based on ethically-produced statistics are likely more reliable and just, leading to better outcomes in policymaking, research, and many other fields. This underscores the significant role of ethical standards in statistical practices, as they contribute to the betterment of society by ensuring data-driven decisions are grounded in principles of integrity and respect.

2.3 Are current processes and protections sufficient?

GDPR and the Data Protection Act 2018  represent comprehensive legal frameworks established by the European Union and the UK. These laws are designed to protect individual privacy rights in the context of data collection, storage, and processing. These regulations stipulate several key requirements for organisations dealing with personal data. Article 35 of the GDPR, for example, mandates data protection impact assessments (DPIA) when there is a likely high risk to people's rights, especially when using new technologies. This process helps organisations identify, assess, and mitigate privacy risks.

Moreover, it is also good practice to perform a DPIA for any use of personal data, even if the regulations do not explicitly require it. This promotes transparency and a proactive approach to potential data protection issues. The regulations also emphasise the importance of accountability and documentation. As per Article 5(2) of the GDPR (the accountability principle) and Article 30, organisations must comply with data protection laws by documenting their data processing activities. Furthermore, data analysis or automated decision-making processes should not result in discriminatory outcomes, as defined in the Equality Act 2010. Organisations must be aware of and take steps to mitigate potential biases in their data or algorithms that might lead to unfair treatment or discrimination.

The measures outlined in the regulatory framework are comprehensive, offering significant protections for individual privacy rights. However, it is also worth noting that the effectiveness of these protections is dependent on their implementation and enforcement. Regular audits, reviews, and updates are crucial to ensure these protections keep pace with technological advancements and emerging challenges. While these processes and protections under GDPR, DPA 2018, and other related laws provide a strong foundation, it is essential to recognise that data ethics extend beyond mere legal compliance. Organisations should continually strive for ethical data practices that consider the potential societal impacts of their data use and uphold principles of fairness, transparency, and respect for individual rights. In an era of rapid digital innovation, the dialogue around what constitutes sufficient data protection must evolve.

August 2023