Written Evidence Submitted by the Informtion Commisioner’s Office (ICO)
(GAI0112)
About the ICO
1. The Information Commissioner has responsibility in the UK for promoting and enforcing the UK General Data Protection Regulation (UK GDPR), the Data Protection Act 2018 (DPA 2018), the Freedom of Information Act 2000, and the Environmental Communications Regulations 2003 (PECR), among other legislation.
2. The Commissioner is independent from government and upholds information rights in the public interest, promoting openness by public bodies and data privacy for individuals. The Commissioner does this by providing guidance to individuals and organisations and taking appropriate action where the law is broken.
3. The Information Commissioner’s Office (ICO) has set out its new strategic vision in the ICO25 plan,1 which highlights promoting regulatory certainty and safeguarding the public as key priorities. In that context, the ICO welcomes the opportunity to respond to this call for evidence,2 and to support Parliament in examining the effectiveness of AI governance and the UK
Government’s proposals.
The ICO’s approach to regulating AI and engagement with counterparts
1. While many AI applications do not involve the processing of personal data – such as climate modelling or non-human genetic research – the uses of AI with greatest salience for public policy are typically powered by personal data. This personal data may be processed to design, train, test or deploy an AI system. All these stages of AI development and deployment fall under the ICO’s purview, as the data protection regulator.
1 https://ico.org.uk/about-the-ico/our-information/our-strategies-and-plans/ico25-plan/
2 https://committees.parliament.uk/call-for-evidence/2750/
4. The aim of data protection law is to mitigate risks to the fundamental rights and freedoms of individuals in regard to the processing of their personal data, including cases where this processing is part of the development or use of AI. This includes risks that can lead to physical, material and non-material damage (see Recitals 83 and 85 of the UK GDPR). As such, the ICO plays a central role in the governance of AI.
5. AI is a strategic priority for the ICO. The recently launched ICO253 strategic plan highlights our current work in this area, including actions to tackle urgent and complex issues such as AI-driven discrimination.4 This builds on our existing work on AI, including:
- our landmark Guidance on AI and Data Protection5;
- our accompanying AI and Data Protection risk toolkit6 (which recently won a Global Privacy and Data Protection Award7 for its impact on enhancing accountability);
- our supplementary guidance on Explaining Decisions Made with AI8, co-
badged with the Alan Turing Institute;
- our support for AI innovators through our Regulatory Sandbox and Innovation Hub9;
- our contribution to standard-setting initiatives as a member of the AI Committee of the British Standard Institution (BSI); and
- our supervision of organisations using AI, including through both proactive audits and investigations.
We continue to track developments in AI to ensure that our policy positions reflect the latest technological opportunities and risks, and have recently established a series of post-doctoral fellowships that research issues such as AI and dark patterns, or model inference attacks.10
6. Where circumstances require, we have taken regulatory action to tackle risks from the use of AI in different sectors, including cases such as that against Clearview AI11 and the Scottish Government and NHS National Services Scotland (NHS NSS)12. Broader current areas of focus include the use of AI in the welfare system, targeted advertising, recruitment and higher education.
7. The ICO recognises the role that other regulators play in governing the use of AI in different sectors or context. We have been at the heart of initiatives to
3 https://ico.org.uk/media/about-the-ico/documents/4020926/ico25-plan-for-consultation-20221407-v1_0.pdf
4 The ICO will soon update the fairness component of the existing Guidance on AI and Data Protection with the aim of assisting organisations tackle such issues.
5 https://ico.org.uk/for-organisations/guide-to-data-protection/key-dp-themes/guidance-on-ai-and-data- protection
6 https://ico.org.uk/for-organisations/guide-to-data-protection/key-dp-themes/guidance-on-ai-and-data- protection/ai-and-data-protection-risk-toolkit
7 https://globalprivacyassembly.org/news-events/gpa-awards/
8 https://ico.org.uk/for-organisations/guide-to-data-protection/key-dp-themes/explaining-decisions-made- with-artificial-intelligence/
9 https://ico.org.uk/about-the-ico/what-we-do/ico-innovation-services
10 You can read more about ICO’s work on AI here: https://ico.org.uk/about-the-ico/what-we-do/our-work-on- artificial-intelligence
11 ICO issues provisional view to fine Clearview AI Inc over £17 million | ICO
12 ICO reprimands Scottish Government over need to be upfront about NHS Scotland COVID Status app’s use of people’s details | ICO
foster greater regulatory coherence and certainty for organisations developing and using AI, both as a founding member of the Digital Regulation Cooperation Forum (DRCF) and as the chair of the Regulators and AI Working Group, which includes 27 UK regulatory authorities. As part of our work at the DRCF we have published two discussion papers on algorithmic harms and benefits,13 and the landscape of AI auditing,14 while continuing to build on that work through our 2022-2023 work programme.15
8. The ICO works with regulatory counterparts and stakeholders not just domestically but also internationally to provide consistency in the law, maximise certainty for people on what protections they can expect, and for businesses and organisations on what standards are expected. In that context, the ICO has engaged on AI in fora such as the Global Privacy Assembly (GPA),16 the Global Partnership on AI (GPAI) and the G7 grouping. Two impactful international proposals on a legal framework for AI, the EU AI Act17 and the Council of Europe’s legal framework on AI18 also received input from the ICO.
9. At a domestic level the ICO has worked with the Government on the reforms to AI regulation that will be introduced through the Data Protection and Digital Information Bill19 that was laid before Parliament on 18 July 2022. We support the Government’s policy intent to reframe Article 22 of UK GDPR as a right to specific safeguards, rather than as a general prohibition on solely automated decision-making, and to clarify the provisions around processing for bias mitigation in AI systems.20 We also continue to engage with the Government on its broader proposals for reform of the AI regulatory landscape.21
13 https://www.gov.uk/government/publications/findings-from-the-drcf-algorithmic-processing-workstream- spring-2022/the-benefits-and-harms-of-algorithms-a-shared-perspective-from-the-four-digital-regulators
14 https://www.gov.uk/government/publications/findings-from-the-drcf-algorithmic-processing-workstream- spring-2022/auditing-algorithms-the-existing-landscape-role-of-regulators-and-future-outlook
15 https://www.gov.uk/government/publications/digital-regulation-cooperation-forum-workplan-2022-to- 2023/digital-regulation-cooperation-forum-plan-of-work-for-2022-to-2023
16 https://globalprivacyassembly.org/
17 https://ico.org.uk/about-the-ico/consultations/eu-proposed-artificial-intelligence-act/
18 https://ico.org.uk/about-the-ico/consultations/council-of-europe-ad-hoc-committee-on-artificial-intelligence- cahai-multi-stakeholder-consultation/
19 https://bills.parliament.uk/bills/3322
20 https://ico.org.uk/about-the-ico/consultations/department-for-digital-culture-media-sport-consultation-data- a-new-direction/
21 https://ico.org.uk/about-the-ico/consultations/dcms-consultation-establishing-a-pro-innovation-approach-to- regulating-ai/
ICO responses to the Committee’s questions
Question 1: How effective is current governance of AI in the UK?
10. Our response to this inquiry focuses on the intersection of AI governance with our regulatory remit, which covers the processing of personal data. Personal data is information that relates to an identified or identifiable individual.22 AI systems that process only aggregate or non-personal data (e.g. environmental data) do not fall under data protection law.
11. We consider that the ICO plays a central and effective role in the governance of AI, as set out in the previous section. We scan the horizon for AI risks, provide clear, up-to-date policy and guidance, and support AI innovators to develop their ideas in compliance with the law. We respond to AI-related complaints, conduct audits of AI systems and investigate AI-related cases. We work closely with other AI regulators, with stakeholders and with our overseas counterparts.
12. Many of the risks that AI can give rise to are not novel, but are already addressed by data protection law and reflected in its provisions. For example, high-risk uses of personal data to develop or use AI are already covered in the provisions on Data Protection Impact Assessments (DPIAs). Issues of discrimination are addressed through aspects such as the fairness principle and the technical and organisational requirements in the UK GDPR’s Recital 71.23
13. We believe that the UK GDPR and the DPA 2018, and in particular the foundational, statutory principles of transparency, fairness, lawfulness, data minimisation, purpose limitation, storage limitation, accountability, accuracy and security, are a vital foundation for the responsible governance of AI across sectors.
14. The Government is reforming data protection law through the Data Protection and Digital Information Bill,24 which was introduced to Parliament on 18 July 2022. We support the Government’s policy intent to reframe Article 22 of UK GDPR as a right to specific safeguards, rather than as a general prohibition on solely automated decision-making, and to clarify the provisions around processing for bias mitigation in AI systems25.
15. We continue to engage with the Government on its broader proposals for reform of the AI regulatory landscape and have responded to its consultation on “Establishing a pro-innovation approach to regulating AI”26. We note that
22 https://ico.org.uk/for-organisations/guide-to-data-protection/guide-to-the-general-data-protection- regulation-gdpr/key-definitions/what-is-personal-data/
23 Even though Recitals in the UK GDPR are not legally binding on their own, they are critical to understanding and applying the legislation. They provide additional context to supplement the main provisions.
24 https://bills.parliament.uk/bills/3322/publications
25 https://ico.org.uk/about-the-ico/consultations/department-for-digital-culture-media-sport-consultation-data- a-new-direction/
26 https://ico.org.uk/about-the-ico/consultations/dcms-consultation-establishing-a-pro-innovation-approach-to-
many of the non-statutory principles proposed by the Government are already found in data protection law in some form, and as such the ICO is already able to apply and enforce them. Getting the interplay right between the proposed new principles and the existing data protection framework will be crucial to providing clarity and certainty to business and citizens alike.
16. The ICO continues to engage with Parliament on the present and future of AI governance. In September 2021 the ICO responded to the House of Lords
Justice and Committee’s call for evidence on the use of new technologies - including AI - in the application of the law,27 and provided evidence to the Liaison Committee in the context of the “AI in the UK: No Room for Complacency” report.28
Question 2: What are the current strengths and weaknesses of current arrangements, including for research?
17. We agree with the Government that the UK has a “world leading regulatory regime, known for its effective rule of law and support for innovation” 29. We note, however, the need for enhanced coordination across digital regulators on the governance of AI, to maximise effectiveness and minimise undue burden on business. We continue to engage with the Government on its proposals for reform of the AI regulatory landscape.
18. We believe that the Digital Regulation Cooperation Forum (DRCF), which brings together the ICO, Competition and Markets Authority (CMA), the Financial Conduct Authority (FCA) and Ofcom to ensure effective governance of digital services, is helping address this coordination need. Earlier this year, it published discussion papers on the benefits and harms of algorithms, and on algorithmic auditing, as a step towards fostering greater regulatory coherence. It is also leading work across the four regulators to make it easier for innovators to bring new products to market.
19. We recommend against the introduction of further entities to oversee the governance of AI. The existing governance landscape is complex and includes a range of technical and advisory bodies in addition to regulators. Care is needed to avoid introducing further entities or coordination interfaces that would make the governance of AI less efficient or effective, or create complexity and burden for AI innovators.
20. We acknowledge AI holds great promise in the field of research. At times research can entail the processing of personal data, in which case data protection applies. The UK GDPR includes specific research provisions to enable the processing of personal data for use cases that serve the public interest and promote scientific research. The ICO has published specific guidance on the UK GDPR’s research provisions, which are applicable to AI-
27 https://committees.parliament.uk/writtenevidence/38632/html/#_ftn1
28 https://publications.parliament.uk/pa/ld5801/ldselect/ldliaison/196/19603.htm#_idTextAnchor002
29 https://ico.org.uk/about-the-ico/consultations/dcms-consultation-establishing-a-pro-innovation-approach-to-
driven research,30 and will continue to work towards providing further clarity on how AI development intersects with scientific research.
Question 3: What measures could make the use of AI more transparent and explainable to the public?
21. Transparency is one of the foundational principles of data protection and is fundamentally linked to fairness. Processing of personal data is transparent when organisations are clear, open and honest with people from the start about who these organisations are, how and why they use their personal data.
22. Transparency is fundamental to the ‘data protection by design and by default’ approach required by UK GDPR. Organisations are expected to integrate data protection into their processing activities and business
practices, from the design stage right through the AI lifecycle. Organisations should consider how to make their use of AI more transparent and explainable to the public from the outset, as opposed to after harms or public concerns have materialised.
23. The ICO has provided extensive guidance on transparency as part of our core UK GDPR guidance31 but also as part of our Accountability Framework.32 In relation to AI, the ICO has published guidance on “Explaining Decisions Made with AI”,33 co-badged with the UK’s national institute for data science and AI, the Alan Turing Institute. The guidance articulates why different contexts will require different explanations.
24. Individuals that could be affected by high-risk AI systems should be consulted in the early stages of AI development, as recommended in the UK GDPR34 and ICO’s guidance.35 This would assist in taking account of their explainability needs when building or deploying AI systems.
25. We should not put the onus of oversight on the public itself, expecting them to understand the nuances of AI development and use. However, general transparency standards are necessary so that the public can understand what AI systems are used, where, how and with what safeguards in place.
30 https://ico.org.uk/for-organisations/guide-to-data-protection/key-dp-themes/research-provisions
31 https://ico.org.uk/for-organisations/guide-to-data-protection/guide-to-the-general-data-protection- regulation-gdpr/principles/lawfulness-fairness-and-transparency/#transparency
32 https://ico.org.uk/for-organisations/accountability-framework/transparency/
33 https://ico.org.uk/for-organisations/guide-to-data-protection/key-dp-themes/explaining-decisions-made- with-artificial-intelligence
34 Article 35(9) of the UK GDPR requires organisations that are engaged in high-risk processing to prepare a DPIA and where appropriate seek the views of data subjects or their representatives on the intended processing.
35 https://ico.org.uk/for-organisations/guide-to-data-protection/key-dp-themes/guidance-on-ai-and-data- protection/what-are-the-accountability-and-governance-implications-of-ai
degree accountable, the public cannot contest the decision and ask for a human review using Article 22.
32. Currently, many decisions that have significant and legal effects on individuals are partly automated, rather than solely. The limitations on
people’s ability to contest such decisions can create challenges. In the ICO’s response to the UK Government’s “Data: A New Direction” consultation, we proposed the inclusion of partly (non-solely) automated decision-making45 in Article 22.
33. The ICO has already provided views on what meaningful human review should look like as part of our guidance on AI and Data Protection. We also addressed this in evidence submitted to the House of Lords Justice and Home Affairs Committee’s call for input on the use of new technology in the application of the law.46 We are grateful for the Committee adopting our recommendation that human review of automated decisions should have the following considerations:
- Human reviewers must be involved in checking the system’s recommendation and should not just apply the automated recommendation to an individual in a routine fashion.
- Reviewers’ involvement must be active and not just a token gesture. They should have actual ‘meaningful’ influence on the decision,
including the ‘authority and competence’ to go against the
recommendation.
- Reviewers must ‘weigh-up’ and ‘interpret’ the recommendation, consider all available input data, and also take into account other additional factors.
Question 5: Are current options for challenging the use of AI adequate and, if not, how can they be improved?
34. One of the key prerequisites for challenging the use of AI or the decisions it leads to is explainability and transparency. This is required both in relation to the decision-making process itself but also in terms of the governance around it (e.g. who collected the data, which data was used to train the model, what features were prioritised, how human review interacts with the outcome, etc).
35. The information rights provided to individuals in the UK GDPR are important in that context. We note the information rights around automated decision- making highlighted in our response to Question 4 and the opportunities to enhance these. Continued work is needed to ensure that individuals are aware of these rights and how they can use them.
45 https://ico.org.uk/media/about-the-ico/consultation-responses/4018588/dcms-consultation-response- 20211006.pdf
46 https://ico.org.uk/about-the-ico/consultations/house-of-lords-justice-and-home-affairs-committee-call-for- evidence-the-use-of-new-technologies-in-the-application-of-the-law/
36. The UK GDPR gives individuals the opportunity to lodge complaints with the Commissioner47 in the case that an organisation fails to provide them with information about how it handles their data. The ICO will maintain a watching brief on the uptake and effectiveness of information rights as AI adoption grows.
Question 6: How should the use of AI be regulated, and which body or bodies should provide regulatory oversight?
37. The majority of AI systems that will impact individuals will need to process personal data either in the development or the deployment stage. Organisations can use the personal data of one group of individuals to train an AI model but then deploy it in a different context and on different groups. In both cases personal data more likely than not will be used. The UK GDPR and the DPA 2018 provide a legal framework that oversees the processing of personal data, so AI is already regulated to a substantial degree.
38. We believe that the ICO should continue to play a leading role in regulating AI in any reformed governance landscape. We continue to promote regulatory coherence, cooperation and capability-building through our bilateral relationships (enabled by Memorandums of Understanding48), the DRCF and the wider (ICO-hosted) Regulators and AI Working Group. We also maintain strong partnerships with other governance actors, such as the Centre for Data Ethics and Innovation and Alan Turing Institute.
Question 7: To what extent is the legal framework for the use of AI, especially in making decisions, fit for purpose?
39. We believe that data protection law provides a robust framework for the governance of AI that uses personal data to make decisions that will affect people, the economy and society.
40. Some indicative – but not exhaustive – provisions of data protection law include:
- Solely automated decision-making: Article 22 of the UK GDPR prohibits the processing of personal data to make solely automated decisions that have legal or similarly significant effect on individuals. There are limited exceptions to this prohibition,49 which require organisations to provide individuals with the ability to challenge the decision and to ask for a human intervention. Recital 71 also asks
47 https://ico.org.uk/make-a-complaint/data-protection-complaints/what-to-expect/
48 https://ico.org.uk/about-the-ico/our-information/working-with-other-bodies/
49 For example the provisions of Article 22 do not apply if the processing is authorised by domestic law, based on explicit consent or necessary for a contract.
organisations to put in place technical and organisational measures to avoid solely automated decision-making with discriminatory effects.
- Fairness50 principle: At a high level, fairness means that private and public organisations should only handle personal data in ways that people would reasonably expect and not use it in ways that have unjustified adverse effects on them.
- Accountability principle: The accountability principle makes organisations processing personal data accountable for complying with data protection law and for demonstrating that compliance in any AI system that processes personal data.51
- Data subject rights: Under data protection law individuals have a number of rights relating to their personal data including the right to access it, challenge its accuracy and its use in different contexts. Within AI, these rights apply wherever personal data is used at any of the various points in the development and deployment lifecycle of an AI system.52
41. We believe that the current legal framework is fit for purpose; however, there is always room for improvement. Providing more safeguards for partly (and not only solely) automated decisions could make the existing AI governance landscape more robust.
Question 8: Is more legislation or better guidance required?
42. In our practical experience, most AI use cases the ICO has engaged with to date highlight challenges in the interpretation of existing law rather than specific legislative gaps. We believe that in the first instance close monitoring, up-to-date guidance and close cooperation between regulators is required. The DRCF is an important development in this regard, facilitating regulatory coherence, cooperation and capability-building across its members.
43. With rapid technological development, AI regulation needs to remain agile: horizon-scanning for future risks and amenable to review when evidence proves gaps do exist. The ICO and other UK regulators have launched foresight teams to identify future regulatory risks and opportunities and design appropriate regulatory responses. For example, our recent Foresight report on biometrics set out ICO’s expectations and concerns around the use of emotion recognition technology.53
50 https://ico.org.uk/for-organisations/guide-to-data-protection/guide-to-the-general-data-protection- regulation-gdpr/principles/lawfulness-fairness-and-transparency/#fairness
51 https://ico.org.uk/for-organisations/guide-to-data-protection/key-dp-themes/guidance-on-ai-and-data- protection/what-are-the-accountability-and-governance-implications-of-ai/
52 https://ico.org.uk/for-organisations/guide-to-data-protection/key-dp-themes/guidance-on-ai-and-data- protection/how-do-we-ensure-individual-rights-in-our-ai-systems/
53 https://ico.org.uk/media/about-the-ico/documents/4021971/biometrics-foresight-report.pdf.
Question 9: What lessons, if any, can the UK learn from other countries on AI governance?
44. The ICO is watching developments in terms of AI governance in other jurisdictions closely. We are a member of the AI Working Group at the Global Privacy Assembly, and we engage closely with G7 data protection and privacy authorities. We have provided input to a range of international proposals on AI governance, such as the EU AI Act or the legal framework under development by the Council of Europe, with the ambition of fostering alignment with the regulatory approach taken in the UK.
45. We support the growing consensus across the world, as demonstrated in the OECD54 and G20 principles on AI55, that individuals’ rights and freedoms should be at the centre of AI governance, as they are at the centre of data protection56. We note that several European data protection authorities are preparing to take on an increased remit for the oversight of AI under the EU AI Act, given the intersection between AI and data protection matters.
Conclusion
46. We welcome the House of Commons Science and Technology Committee’s inquiry into key AI governance issues, such as the impacts of biased algorithms, the lack of transparency on how AI is applied and how automated decisions can be challenged. Public trust is paramount for the wide adoption of AI in the UK and scrutiny of the current and future governance of AI is crucial.
47. The ICO has been playing a leading role among the UK regulatory community in terms of developing its thinking on AI, forging alliances with UK and international stakeholders and supporting innovation57 that relies on personal data while upholding the fundamental rights and freedoms of the people what provide it. We hope our submission is helpful for the Committee and remain open to discussing these issues in more detail.
(December 2022)
54 https://legalinstruments.oecd.org/en/instruments/OECD-LEGAL-0449
55 https://www.g20-insights.org/wp-content/uploads/2019/07/G20-Japan-AI-Principles.pdf
56Article 1 of the UK GDPR explicitly states the regulation “protects fundamental rights and freedoms of natural persons and in particular their right to the protection of personal data
57 The ICO is supporting innovators via a range of services including the Innovation Hub, the Regulatory Sandbox and the upcoming Innovation Advice service.