Professor David Leslie and Ravi Naik.

Q1 The Chair: It is my pleasure and privilege to welcome you to the 26th meeting of the Joint Committee on Human Rights. As the name implies, we are a committee that is made up of Members of both Houses of Parliament: six Members of the House of Commons and six Members of the House of Lords, drawn from different political traditions and backgrounds. We have a mandate to look at the impact of legislation and policies on human rights in the UK and on citizens resident here, whether they are British citizens or not.

Our recent inquiries led to two legislative reports, one on the Border Security, Asylum and Immigration Bill, which is now at its Committee stage in the House of Lords, and our report on the Mental Health Bill, which is being considered in the House of Commons. We are currently also taking written evidence on the Crime and Policing Bill. We are coming to conclusions on two of our major thematic inquiries. One is on transnational repression and the other one is on supply chain transparency and the use of coerced or slave labour. If any of those who are watching our proceedings online want to know more about the committee’s work, please go to our website where our report on the Border Security, Asylum and Immigration Bill is now available, nearly 100 pages of it.

Today the committee is going to explore some concepts that underpin artificial intelligence and their implications on human rights. Our inquiry will explore what regulation is required for the use of AI to safeguard human rights in the United Kingdom in order to inform any legislation or measures that the Government plan to introduce. It may be of help to our witnesses, but also to those who now follow the proceedings on this inquiry, to know the draft terms of reference. They are: how can AI affect human rights for good or ill? How much difference will the framework convention on artificial intelligence make? How far does the UK’s existing legal framework provide protections for human rights in relation to AI? Is the Government’s policy approach to deploying AI in its AI Opportunities Action Plan sufficiently robust in respect of safeguarding of human rights? How do the UK’s plans to regulate the impact of AI on human rights compare with those of other jurisdictions such as the European Union? What would be needed in any future legislation to protect human rights? How far should AI’s use by public and private bodies be treated differently?

Well, to explore those issues with us, we have two eminent witnesses. We have Ravi Naik, who is the legal director of the law firm AWO. We have Professor David Leslie, professor of ethics, technology and society at the Digital Environment Research Institute at Queen Mary University of London, and director of ethics and responsible innovation research at the Alan Turing Institute. Let me say a word more about both our witnesses before turning the first question to them. Ravi Naik is an award-winning solicitor and co-founder of AWO, a pioneering practice working on data protection, AI and advanced digital technologies. Professor Naik has acted on several of the leading cases about digital technology. His advisory practice includes a range of clients, including regulators, unions, technology companies and high-profile individuals. He is described in directories as, “The best of the best solicitors to work with on bleeding-edge tech and data issues”, and “Ahead of the curve” is one of the other quotations I saw—“An innovative thinker and a joy to work with”. He is also an honorary professor at UCL. Welcome.

Professor David Leslie, professor of ethics, technology and society, is the author of the UK Government’s official guidance published in 2019 on the responsible design and implementation of AI systems in the public sector, Understanding Artificial Intelligence Ethics and Safety and principal co-author of Explaining Decisions Made with AI, published in 2020. He is co-badged as guidance on AI Explainability, published by the Information Commissioner’s Office and the Alan Turing Institute. He is an organiser of the Human Rights, Democracy and the Rule of Law Impact Assessment for AI Systems project, which aims to support the Council of Europe Framework Convention on Artificial Intelligence and Human Rights, Democracy and the Rule of Law. It is worth saying in parenthesis that some members of this committee recently visited Strasbourg and we talked directly to members of the Council of Europe around these questions.

I will ask the first question, before turning to my colleague Sir Desmond Swayne. What would you say are the key features of artificial intelligence that make it different from other technologies? Perhaps you could outline some uses that are already being made with AI and have human rights implications. Thank you.

Professor David Leslie: Thank you so much, it is a really great honour to be here and a pleasure to be able to talk about this content with you. When we think about the exceptional feature of AI vis-à-vis other technologies, it is that these systems are serving surrogate cognitive functions in the world; so they are standing in for the behaviour of thinking and acting human beings without, at the same time, having the same mechanisms of accountability and interpersonal responsibility that come along with making decisions, acting in the world. You have this interesting phenomenon where AI systems are affecting people in both good and negative ways, without the systems themselves bearing the right type of responsibility for the outcomes at the same time.

Just to say, if we were going to have a roundabout definition of what AI is in this respect, we would think of these systems generally as an ensemble of computational or algorithmic techniques that serve functions in the world that would otherwise require human intelligence. We talk about this as a functional definition: it is a way that we can talk about the technology without focusing on some of the details of the actual machinery of the technology. That being said, that type of definition is also limiting in a way because when we think about AI, AI technologies are sociotechnical systems. So when we think about the technologies, they are the results of human choices, human decisions and human practices, and we have to think of the technologies in a very sociotechnical, life-cycle-oriented way. Right from the very beginning of design choices about how we scope a project, what solutions we are looking to provide and what problems we are seeking to solve, these are patently human deliberations and decisions.

If you move through the life cycle, even when you set a target variable—so you find a way to quantify your goals—that again is a human decision, and we need to scrutinise each decision point from the very first moment of the design through the development of the system: setting the type of modes of validating the system, how we benchmark the system, what are the performance metrics that we use to measure the system? These are all significant human choices; we say sociotechnical choices.

One essential element here—I am mentioning this because this has direct bearing on human rights—is that, when we think of an end-to-end stream of accountability or an end-to-end stream of transparency, we are looking for transparency of the practices and choices. We are not talking about necessarily just the transparency of the technology itself; we want to understand what decisions were made? Were they sufficiently bias mitigating? Did they have sufficient regard for the safety, reliability and robustness of the system? Were they sufficiently anticipative about the effects on freedom of expression et cetera? So when we are looking at these technologies, we are looking at them as these sociotechnical practises.

I will maybe make one more comment at this general level: that functional definition that I offered at the very beginning is a nice guide for us to think about how we identify the systems out there in the world, but we also need to take a broader perspective on how to define the technology. The broader perspective is let us think about what the enabling conditions are: what are the preconditions of having the technology actually out there in the world? If we think in this way, we are thinking not just about a limited series of decisions in a technical life cycle but about the entire AI value chain. That is absolutely essential right now because AI is constituted not just by the decisions that go into it; it is constituted by a long stream of data that is being used to train the system, the compute or information-processing power that is needed to train the system and operate the system and those oftentimes hidden efforts of ghost workers who are vetting the systems for toxicity. There are many layered interrelated decision points and elements along this very complex value chain that are needed to have the AI systems out there in the world.

We should not limit ourselves to a functional understanding; we have to think about this broader value chain perspective. You mentioned this idea that slave labour and understanding the inner workings of value chains are important in human rights and they absolutely are, especially with AI.

The Chair: That is a very helpful introduction, Professor Leslie; thank you for that. It sets the scene very well for us and I am very struck by the dangers of reinforcing discrimination and inequalities, but what we put into the system is what we are going to get back out of it, so we want to learn more about how that works as we proceed. Professor Naik, would you like to add to that and give us a curtain raiser?

Ravi Naik: Thank you to the committee for inviting me here and for giving me the opportunity to explain some work and provide evidence to your very timely inquiry. I will start by saying I definitely agree with what Professor Leslie has said, and there are two things coming from it that are worth emphasising. First, when we think about AI and your question of what makes it different, we have to think about what model we are talking about because these models can do different things. For example, models that can generate content and predict words are very different to predictive AI that can make a prediction like a person, which may have a totally different risk matrix.

What might be quite helpful at this point is to stand back from your question: what makes things different? From a legal perspective, we quantify differences by definitions. Definitions are used to do two things: to specify something but to distinguish it from others. To that end it seems quite simple, but you think about all the attempts to create a definition of AI and there is no singularly accepted definition. That speaks volumes to what it is we are trying to specify, draw the red lines around, as well as what we are trying to distinguish it against. To that end, the OECD has done a fantastic job of explaining this exact problem in the definition in saying, “Well if you try to distinguish it a few issues arise. The technology develops very quickly”. One clear example it gives is OCR, optical character recognition, which used to be considered textbook AI but is no longer because it does not go to this new kind of zeitgeist world that we live in of generative AI, predictive AI and so on.

I will move on from that to say a few other things that are important in this definitional concept. One is, in the UK context, we have various inconsistent definitions of AI in different bits of legislation, which is interesting and quite striking. Sorry, I can hear a bell; do you need to stop me?

The Chair: I would not like to stop you, but you can hear the bell ringing, which means that our colleagues in the House of Commons have to go and vote. I am going to suspend the sitting while the Division takes place, and I hope that those watching online will understand why we are suspending the hearing and will rejoin us as soon as the Division has been taken.

Sitting suspended.

The Chair: I am happy to invite you back to the resumed meeting of the Joint Committee on Human Rights where we are considering artificial intelligence and its impact on human rights. We were listening to Professor Naik before we were interrupted by a Division in the House of Commons. My colleagues are now back in their places. Please continue your evidence to us.

Ravi Naik: Thank you. I was in the middle of talking about definitions and going to your very helpful question of what makes AI different. My position when we broke was talking about the OECD definition, and I can read out some of the main features—echoed in the EU’s AI Act—as a good starting point of the definition: the ability to infer, autonomy and adaptiveness are some key things coming from those definitions. But there is no agreed definition and, notably in UK law, there are different and inconsistent legislative provisions defining AI, which as a starting place is a problem.

Another definitional consideration that we have as part of this is what we mean by human rights. Which rights are we talking about? Are we talking about the rights as those contained in the Human Rights Act or public law principles and how they might apply across? Are we talking about, for example, data protection as a human right, as it is recognised in the European charter of human rights? Obviously, the European charter of human rights is not part of our law, but the fact that data protection is recognised and has that elevated status means that the Europeans have a different approach to data protection than we currently do in law.

Likewise, what do we mean about human rights against private actors? As we all know, AI is a space dominated by private actors rather than public authorities creating and deploying this technology, so what do we mean about human rights against private actors? A third level when we are thinking about differentiation, definitions, is commonality of language. I will give you one example: we have talked a lot about bias, which probably means very different things to human rights lawyers, data scientists and statisticians, so having an agreed glossary of terms when we are thinking about all three of those concepts is really important. I see the Parliamentary Office of Science and Technology has taken this approach: there is a glossary of AI terms which I find really useful in my own work and I would encourage the committee to look at that glossary and maybe take a similar approach.

The Chair: That is a very helpful observation; thank you. Looking at what constitutes bias and what is not, obviously, is essential to the kind of view we will take about the impact that AI can have, especially on groups who are most likely to be discriminated against. I am going to turn now to Sir Desmond Swayne, and after that we will hear from Mr Alex Sobel, Member of Parliament.

Q2 Sir Desmond Swayne: In George Orwell’s Nineteen Eighty-Four, his dystopian vision of the future was of technology harnessed to deprive us of our privacy, our liberty and our autonomy. About 20 years previous to that publication, in I think 1931, Aldous Huxley produced a very different vision of the awful future where actually it is our adoration of technology and how we conspire with it to be deprived of our ability to think. In Neil Postman’s book Amusing Ourselves to Death, he defines the difference between the two visions as being in Nineteen Eighty-Four information is denied to us, but with Aldous Huxley you are swamped with it so that you become completely passive.

In Orwell’s view, books are banned; well there is absolutely no need to ban a book in the Brave New World because no one would possibly want to read one. The truth is concealed in Nineteen Eighty-Four but actually you are provided with so much information in the Brave New World that it is lost in a sea of irrelevance. In Nineteen Eighty-Four you have a captive culture whereas we have an utterly trivial culture. So with respect to AI, which is it to be? Or is it going to be the worst of both? Let us start with you, Professor Naik.

Ravi Naik: This goes back to what Professor Leslie was saying: you have to think about the chain and the uses because these systems were not just handed down from on high; they were created, they were developed and they are being used by individuals in different circumstances. There is the idea of information overload: in some ways it is fantastic that you can go on to a search engine and get an answer, and with the introduction of AI into those search engines you get those answers potentially quicker. But the problem becomes knowing whether that information is correct, whether it is trustworthy and whether it is something you can rely on. In my own profession, lawyers have started using AI to rely on cases that are not real, and that has caused all sorts of problems for the profession: there is a database recording how many times lawyers have relied on fake cases. So that is an example, I guess, of the information overload and trying to see the good through the bad.

The alternative of George Orwell’s Nineteen Eighty-Four and the tyranny that it might create again comes down to use. In the Nineteen Eighty-Four example, it was the Government, state agencies, using technology to mandate and prevent freedom and liberty and libertarian ideals. But if Governments across the world start to use this quite sophisticated technology, one of the key things we like to focus on in our work is, if something goes wrong, how you have accountability for it. For example, in a military context or even just a day-to-day context, an individual trying to interact with the Government for welfare cheques, immigration status, to pay their taxes or whatever might do so in quite a rudimentary way. How do you have transparency over that? How do you have redress?

Those three things—accountability, transparency and redress—are, for me, the model. What is the model doing? Who is using the model? Where in the model is the supply chain? We can talk about what those things mean. Where is the oversight before development, ex ante, before it is used? What oversight do you have during its life cycle because this technology develops so quickly? How do you have auditors able to have the technological sophistication, the resources and the knowledge? For example, if you have a regulator looking at a technological system in a welfare context, how would it have the ability to properly scrutinise how that technology is being used? Or would it just assume that it has been given this technology so it should rely on it and defer to it? And thirdly, how do you have redress and accountability? That is one of the key fundamental issues to do with human rights: how do you have an effective remedy against something that may change our lives by information overload or other dystopic ways?

Professor David Leslie: I will pick up on one element that you were talking about in terms of the ex ante approach. We are sitting around this table today thinking about how we can marshal human rights as a metric of assessment to measure the impacts of design development and the use of AI technologies on individuals, social relationships and society in general. What this first and foremost should tell us is that we are concerned with bringing in a kind of society-led, or even a human-centred approach to this technology, where we can get out ahead of what would seem like features of history that are deterministic. There are a lot of people who see the so-called inevitability of the emergence of this autonomous technology as if it is going to crop up beyond our historical choices. That is clearly a kind of mistaken perspective in the sense that, first and foremost, humans are always behind the design of the technologies that affect them.

That being said, from that perspective we can look at these specific dystopic scenarios and think about the individual level, the social or interactive level, the interpersonal level, and the ecosystem or societal level. At the individual level, from a human rights perspective we think a lot about human dignity and individual autonomy and the ways in which these systems can, just as you suggested, kind of de-skill or create conditions of cognitive atrophy because ultimately, once you start sort of using a language model that will support you in your everyday cognitive behaviours without lots of forethought and care, it could be that you become overreliant on those technologies, or even overcompliant on the recommendations or outputs that come from those technologies. So managing the transformative effects, potentially, of generative AI and language models, has to involve an ex ante approach that brings to the fore these potential impacts on the kind of formative development of the individual.

We already see, for instance in the education sector, that there is a lot of thinking going on about how we overhaul our methods of pedagogy, of research, so that increasing reliance on these very sophisticated generative AI systems, like Gemini or Claude or ChatGPT, do not displace the agency and creativity that are involved in both learning and research. So at that individual level, we really need to take that anticipatory point of view where it may well be that the horse is out of the barn and the systems are out there already, but it does not mean that we cannot, as a collective, be corrective; there cannot be a kind of remediation of what might get out of hand.

I will also speak about the social interaction level of what you were talking about. It is the case that there are surveillance technologies like live facial recognition and labour management surveillance technologies. Attention analysis technologies that can be used in the workplace to evaluate the engagement of employees in their work environment and how enthusiastic they are. Is their behaviour indicating that they are fully invested in the job? We know that we have to be careful about elements like freedom of expression, freedom of assembly and other ways in which there can be a chilling effect of surveillance technologies on those fundamental freedoms that we have in our everyday lives. There are simply assumed conditions of our interaction in the everyday world that we have to make explicit in order to manage the transformative effects of a technology. This is already happening; surveillance technologies have been almost a frontier technology when it comes to the assertion of rights against it. We have already seen actions here in the UK and in the US to try to curtail the surveillance technologies and we have seen red lines drawn in certain places about the use of live facial recognition, for instance. So we see the intervention having a real-world impact.

I will lastly mention, which I think is the most dystopic scenario, this living in a world where it is not just information overload, but a pollution of the information ecosystem by synthetically generated content, which undermines public trust in the integrity of that information ecosystem. We are now living in a world in which these generative AI systems are increasingly capacitated to simulate, in a very indiscernible way, human communication. For instance, there are ways in which these systems are being enhanced by other data points and information on people that allow these systems to hyper-personalise and produce particular information that will speak to the demographics of a particular user. When you are starting to live in an information ecosystem where you have lots of synthetically generated content that becomes indiscernible, if you will, from everyday human communications, it can undermine democratic processes; it can undermine one’s capacity to fully participate in the moral, political, and social life of the community. This, if anything, is a deep human rights concern and, in bringing to the fore issues of democracy and rule of law, the Council of Europe is really seeking for us to think about these things.

Ultimately, when we think about the integrity of the information ecosystem, we are thinking about the importance of informational plurality. When we transact with each other, we want to have the full range of views presented before us as a way that allows us to enter into the so-called marketplace of ideas and come up with stronger views about what is right and what is wrong in the world. And when you have a tendency towards the homogenisation of information because you have just a handful of proprietary models that are generating a lot of content, you are ultimately going to potentially adversely affect informational plurality. That is something that really needs to be thought about seriously.

The other element that I will mention here in terms of the information ecosystem or ecosystem level is one very concerning dimension of where we are now as opposed to where we were in November 2022 when ChatGPT was launched. In the past few years, what we have seen is this increasing influx of synthetically generated content into the data sphere—if you will. That means that, when you start to train models on web-scraped data, that downstream training data from 2022 onwards will have been subject to what we would think of as data pollution; so it has some human output, it has some machine output and oftentimes all of that output is not discernible. If you are just web-scraping and trying to train systems, you will train on both synthetic and humanly generated content. Studies have shown that when you have a lot of synthetic generated content used as training data, there are phenomena like model collapse where what happens is, as you train a model on more and more synthetic data, the long tail drops off and the model will go to the mean, so you have less of the range of views. That is a technical outcome of training on already statistically generated series of strings of tokens.

The point is that, as we move into a world in the future, we might want to develop better language models with data that has more data integrity. Right now, we would not be in a position to do that because there is just so much more of the synthetically generated content out there. So these are big concerns.

The Chair: Those are big concerns. Thank you for setting the stage, and for helping us to understand this dystopia. Members of the committee will want to further explore with you the safeguards we need to put in place as we confront that brave new world.

Q3 Alex Sobel: In terms of looking at dystopian literature—taking the theme from Desmond’s question—Philip K Dick’s classic Do Androids Dream of Electric Sheep? spawned the sci-fi classic “Blade Runner”. It took the idea of artificial general intelligence housed in a cybernetic body to emerging new technologies. It was set in 2019; in 2025 we are effectively already in the future.

The central theme of the film was: do those beings have rights? We are not there yet, but you both work to apply human rights approaches to new technologies like AI, cyber et cetera, which will collide together in the future. Can you take us through what you do, the approach you take, how you do it and how that informs us here and the public more generally?

Ravi Naik: Do robots have rights? I do not know if I should start there. My perspective is more from the human angle, but there is a very good book by barrister Jacob Turner called Robot Rights, which goes into this in detail.

When an individual comes to you with a concern about something that has happened to them, there are two main issues that arise. First, we always try to understand their objective; what do they want? Secondly, most people do not understand the ins and outs of technology and they do not need to know about that; they need to know why a decision was reached, how that decision affected them, what rights they have and what accountability is in the system.

I am very conscious of the limits of sub judice in this room so I cannot get into too much detail about specific cases, but I will try to speak in the abstract. We have had cases where we have had to interrogate and scrutinise algorithmic systems and the systems that have made decisions. The discovery process is often very illuminating because you can find out the chain of actors involved. It is vital to understand where decisions were made; understanding how you can get accountability for those decisions is crucial. One of the things that drives a lot of our work is how to get an outcome, how to get a remedy for this person who has been wronged. It is always about righting a wrong, and there are two areas in which this sadly becomes a decreasing possibility for people.

First, the human rights framework pertains to public actors, and often the decisions we are talking about are caused by third-party private actors—technological companies that control the technology, control how it is used, and tend to keep a lock on how decisions are made. I would very strongly urge this committee to consider the ability of regulators to look at how systems are made and why decisions are being made. Many systems are said to be proprietary and covered by trade secrets legislation; however, that is often a smokescreen because the models are publicly available through open source, for instance, through a platform called Hugging Face. Regulators may have the power to investigate, but they also need to have the skills to look at technology. It cannot be incumbent on individuals to take such action.

The second issue is cost: the cost of bringing action in this country, particularly against a private actor, is prohibitive for most people. We are talking seven-figure sums to bring action against a private actor just to uphold your rights, let alone seek damages. That has to be a key barrier to accountability in this space. If a government actor has committed a human rights wrong, most people will get legal aid to prosecute their case. The legal aid system exists to make government accountable; that is the very purpose of it, and it is a really powerful tool to ensure accountability. The reason legal aid is extended to government actors was a question of power, but I would question who has power now. In the digital space, a lot of power resides in private hands. So, when we are thinking about how to try to redress a wrong, those are some key concepts and hurdles that are before us.

Q4 Baroness Lawrence of Clarendon: What potential is there for an application of AI technology to promote or undermine individual human rights? Which rights, if any, are the most likely to be affected?

Professor David Leslie: I will start with the last question and work towards this one because we need to get on to the table a topic that should be considered by the committee: the term that Alex Sobel used—artificial general intelligence. I will speak as a researcher; I told Ravi outside that this means I am going to say strongly worded things.

We have to be careful about how we receive the offering of people in the tech world or tech industry who sell us notions of artificial general intelligence. In the research community, from a scientific point of view, those who have scrutinised claims of AGI—artificial general intelligence—have generally seen these claims as unscientific, vague and misleading. When we talk about so-called sparks of AGI, we are talking about statistical models of incomplete or partial linguistic data sets. These are text corpora that have been modelled through a particular technique called the transformer model, which is a deep learning technique. These systems are concatenating or putting together strings of words to predict the next word or next series of tokens. In a deep sense, that is not what we would recognise either as intelligence or general intelligence; while the scaling of data and the scaling of model size has produced some really very useful mechanisms within the system, the system itself is not suddenly assuming an identity that bears rights. The system itself is a heuristic or statistical tool—a technique.

In the research, we talk about how when people reify these techniques and give them agency or talk about this as the emergence of some type of autonomous, independent, superior intelligence, they are falling into the fallacy of misplaced concreteness. They are taking a scientific model, which has always been a scientific model, and they are imputing an existential or ontological status to that model. We know these are statistical systems. They are very efficacious and we need to be very careful about how we use them in the world. But ultimately, agency does not pop out of the ground like a mushroom; these systems do not suddenly become rights-bearing individuals. They are just mathematical techniques; they are hardware run on software.

Alex Sobel: I do not think I was suggesting that we have AGI now, but that it might emerge in the future. Are you saying there is no chance that it will ever emerge?

Professor David Leslie: What I am saying is that the way we talk about AGI is problematic. When we think about how we define intelligence, even how we would define general intelligence, we really need to be sufficiently interdisciplinary. If you talk to cognitive scientists, neuroscientists, developmental psychologists, anthropologists, evolutionary biologists or linguists, and you ask them what general intelligence is, they give a very rich account of the elements that are part of what all human beings and other biological creatures are involved in through intelligent action in the world. It is very different when you think about a mathematical model trained on a small slice of linguistic experience.

When we learn in the world—all of us—and when we come to a cognitive agency in the world, we are coming to that cognitive agency through a tremendous amount of embodied, enacted and lived experience. All those dimensions that give us common sense and the ability to have practical judgment are simply not picked up in the small slice of linguistic data that people are talking about as having sparks of artificial general intelligence. I caution against it. I do not think any of us should be in the business of prediction, but I do think we should be in the business of looking at things in a very critical way. The race for AGI is taking us into a very irresponsible dash to build data centres all around the world without sufficient consideration of biospheric consequences and general societal impacts.

You also have a largely big tech-dominated, “Silicon Valley orthodoxy” as Eric Schmidt calls it, and the emergence of a perspective that ultimately orients us to saying, “If we do not get AGI then China will get AGI”. For that reason, you have to take away all the reasonable regulatory and governance barriers because we will fall behind in the race for AGI. There are big stakes to that discourse.

Alex Sobel: To be clear, I am not a proponent of AGI; I am a sceptic of AGI. It needs heavy regulation.

The Chair: I am anxious that we do not lose sight of the question that Baroness Lawrence was asking about the promotion or undermining of individual human rights. Perhaps, Professor Naik, you might come in.

Ravi Naik: Thank you, Baroness Lawrence; it is a fantastic question and there are a few different elements to it. First, when we talk about rights, as I said at the start, what rights are we talking about? If we take it as a given that data protection is a right, you can imagine automation being used to foster those rights. You can imagine automated requests to erase information, or to object to certain uses of information. You can imagine tools being developed that make access to justice cheaper, swifter and more direct, particularly between technology companies. If you could automate your rights to erasure or to object against a technology company in a particular way, you can imagine doing that against AI; you can see that happening. Maybe on a more rudimentary level, basic claims could be done through automation, which would make a big difference. Those are the positives of automation.

I can see the hindering of rights also being a big issue. Professor Leslie said that none of us here are in the business of prediction. Sadly, there are companies that are in the business of prediction, trying to predict how we might all behave, how we might commit crimes or what our risk of absconding and so on might be. You can see this technology being introduced on an increasing basis in public spheres. That produces a significant risk; I am sure Professor Leslie can talk about this better than I can.

The ability to predict is inherently problematic. It is very hard to predict how a human is going to behave or what risks they might pose. Automation may definitely hinder our rights, particularly if we use the wrong type of technology in the wrong way or use technology for things it is not actually able to do. Professor Leslie’s earlier point on realism about technology and approaching technology from a realistic perspective as to how it will be used and what it can do is a really important component of this discussion.

Baroness Lawrence of Clarendon: My question was asking which rights, if any, are the most likely to be affected by AI.

Ravi Naik: That is why I come back to this question: which rights do we mean? If you go to the Human Rights Act and the classic definition of those first-generation rights, a good one might be the right to an effective remedy in the European Convention on Human Rights; that could be positively influenced, and I feel passionate about it. Data protection might be another one that could be positively influenced.

If we think about negative influence, the right to privacy definitely gets a lot of attention in this space. We do not have a direct right to privacy as in the US, but Article 8 of the European Convention on Human Rights is akin to a privacy right. It has some interesting bite, particularly against public authorities, but the tortious misuse of private information that has arisen from Article 8 of the European Convention on Human Rights has had bite across to private actors as well. We have run cases where we have used those rights against private actors.

Because the Human Rights Act is focused on public authorities, it might have a narrower focus both on the positive and the negative. But if we start to expand the idea of rights out to private actors, then what rights are we are talking about? For example, fairness is a concept in the public law panopticon of common law. Fairness is also a principle within the data protection framework. You have a right to have your data processed fairly; what does that mean in an AI context? You can imagine the positive use of AI to foster the fairness of your rights or the fairness of processing your data because you could ask for your representations to be considered using AI. You can make representations via AI. Is that a principle of fairness we want to bring across from Europe to the UK? Those are the types of basic rights.

Baroness Lawrence of Clarendon: So it is even more complicated to balance and understand where the AI starts and finishes and where our human rights are.

The Chair: We are going to move on to regulation.

Baroness Kennedy of the Shaws: This is one of the key things that is going to be facing us; how do you deal with private actors as distinct from public actors? You cannot; the big corporates have huge, vested interests and refuse to be regulated.

The Chair: Absolutely—let us move on to ask those questions of our witnesses. How do we regulate this? We will turn to Baroness Kennedy and, after her, hear from Dr Peter Swallow.

Ravi Naik: Professor Leslie, do you want to come in on this?

The Chair: We have heard quite a lot on that.

Q5 Baroness Kennedy of the Shaws: Sometimes it is better if just one of you answers from a practitioner’s position, then we benefit from the intellectual and academic in a different way.

I want to home in on the business of regulation because it is the subject of so much current discussion. I was interested in the concept that Silicon Valley’s orthodoxy is that there should be no regulation otherwise we are going to be outstripped by China. Of course, that is used among many other arguments for saying why there should be no regulation—it is going to in some way crush inventiveness and so forth.

I wanted to know what you thought was possible with regard to regulation in the future development of AI. We could go down the road of looking at general intelligence or looking at autonomous systems, but what about safeguarding human rights? Can a regulatory body effectively deal with that, and could law be drafted in a way that made that possible? Professor, I see you making interested nods.

Professor David Leslie: In a sense we are already on a path towards that with the treaty, the framework convention, in the sense that there has now been a six-year effort on the part of the Council of Europe to develop specific approaches to thinking about the governance of AI with regard to its impacts on human rights, democracy and the rule of law.

This also responds to the question that was posed before: there are guidelines for us to think about areas of concern as part of ex ante risk and impact assessment processes that are now in a sense codified in the treaty and made actual or operationalised in the Human Rights, Democracy and the Rule of Law Impact Assessment, but that are part of this journey towards having a better grasp of how to actually enforce existing law. There is equality law here. There is human rights law here, and yet there have been tremendous enforcement gaps.

It is not that there is chaos out there in terms of directions for governing these technologies, especially generative AI. One of the scandals of the last few years is that we have not enforced existing regulation; data protection regulation, intellectual property regulation, anti-monopoly law, and other forms of the regulation of trade could already have had an impact on the way we approach governing the technology.

Baroness Kennedy of the Shaws: You are using the royal “we”. I want to know whether you are talking about we in the UK, we globally, or we in the West?

Professor David Leslie: I am talking about the UK in this situation.

Baroness Kennedy of the Shaws: That is really the second part of my question. What is your view of the UK Government’s current approach to the regulation of AI—indeed, the regulation of the whole of social media et cetera? To what extent is it possible without very deep co-operation with our international partners?

Professor David Leslie: We all know that there has been a long journey to get here. A few years ago, we had the regulatory White Paper, which had the horizontal concepts that were intended to be treated with regard to the vertical regulators and enforcement.

Baroness Kennedy of the Shaws: It is important for you to explain that as the general public are listening. Do you mean horizontal as distinct from vertical in terms of regulation?

Professor David Leslie: The original regulatory White Paper from 2023 outlined a series of general concepts, such as fairness, safety, remedy and recourse. These were considered to cut across all the regulators in the sense that they are general important regulatory concepts that need to be tailored or customised in the particular domain or instance.

Baroness Kennedy of the Shaws: So we are talking about the principles of fairness, transparency and so on, operating wherever there is a regulator, whether it is in terms of water regulation, data regulation or any other area of regulation. There should be fundamental concepts that inform how the regulators ought to behave—if only they did. Anyway, that was the plan.

Professor David Leslie: It might be useful to run through where there has been convergence over the concepts and principles. Over the last five to 10 years, there has been increasing convergence around issues of safety, security, reliability and robustness; around accountability, traceability and auditability; around fairness, bias mitigation and justice; around explainability, interpretability and transparency; and finally around data protection, privacy, quality and data integrity.

Generally speaking, these are realised in one way or another as horizontal principles in the regulatory White Paper, but if you look at the Council of Europe’s framework convention, the seven fundamental principles are safe innovation, reliability, accountability and responsibility, transparency and oversight, respect for privacy and personal data, equality and non-discrimination, and human dignity and individual autonomy. So you can see that there is an intensive overlap. To the UK’s credit, we played a central role in the development of those concepts.

Baroness Kennedy of the Shaws: It is always forgotten that the UK often plays a central role in the development of those legal and ethical principles.

Professor David Leslie: Absolutely. I served on the bureau of the Council of Europe’s Ad Hoc Committee on Artificial Intelligence from 2019, and I have been a specialist adviser to the Council of Europe since 2021. So I have been a first-hand witness of the absolute essential character of having the UK in the room and bringing the knowledge that this country has on such matters to shape these instruments.

Baroness Kennedy of the Shaws: It started in the 19th century when we tried to regulate children being sent up chimneys and working on looms in factories.

Professor David Leslie: The UK made early investments in responsible and trustworthy innovation; we were out ahead thinking about the responsibility of innovation in 2016 and 2017. Reports were written; you had Tim’s report in the House of Lords on AI, and we had the AI sector deal. We founded the Alan Turing Institute around that time, and part of its role has been to advance the public policy dimension of responsible principles. So, with all that said, the UK has had a head start.

Baroness Kennedy of the Shaws: It has pretty much had a leadership role.

Professor David Leslie: That is right; it has had a head start in this global leadership role. Given that context, and with changing geopolitical and geoeconomic conditions, the UK really needs to take a hard look in the mirror and think about how it can retain that leadership role across the globe. The UK was never a fast follower on AI and data regulation. It has always been out ahead of the curve, rather than falling in line with safety discourse and so on. So there are considerations about how we leverage the tremendous amount of intellectual and social capital that we have in this space to retain that leadership role on AI.

The Chair: We have a lot of other colleagues who want to come in on some questions. Thank you very much for the answer. Professor Naik, I would like to bring you in now on the question that was just put to Professor Leslie. I know we have a question on the Council of Europe coming up from Dr Swallow, so perhaps it might be wise to take that at the same time.

Q6 Peter Swallow: Let me just very briefly put on the record that my favourite cultural account of AI and human rights is the “Star Trek: The Next Generation” episode “The Measure of a Man”, in which Data’s human rights are considered.

That aside, we have really helpfully touched there—well, more than touched—on the impact of the Council of Europe’s Framework Convention on Artificial Intelligence and Human Rights, Democracy and the Rule of Law, the important role the UK has had in shaping it and the important role it will have in shaping the UK’s response to AI.

We were approaching one of the key points that we wanted to draw out from today’s session, which was how our response on the impact of AI on human rights compares to those of other jurisdictions. I say this very aware of the context in which different jurisdictions are approaching the wider AI question in terms of the level of risk that they are willing to accept, with the perceptional reality that the EU is more risk-averse, the US is less risk-averse and we find ourselves somewhere in the middle. With that context in mind, how do we compare to other jurisdictions on this important human rights angle?

Ravi Naik: Can I start by answering some of Baroness Kennedy’s questions and then bringing in some answers on the Council of Europe framework? This idea of how to govern is not a new question; it is not something that has not been considered before.

About 12 to 18 months ago, we worked with the Ada Lovelace Institute, which asked us a very simple question: how do you regulate AI in the UK? We said, “That is a very difficult question. Nobody’s ever going to read that report. Why don’t we actually try to use our legal skills for this question that was put to me earlier about process, and actually pass through some hypothetical scenarios? What would it look like if AI were deployed in four different scenarios? Where would the law bite and where would you have some protection?” Rather than read the whole report to you, we put this table together, which hopefully you can all see.

The Chair: Perhaps you could share that with the committee as well.

Ravi Naik: I will. But the reason I am showing it to you here is that it is colour-coded, and there is a lot of red here and not much green.

Baroness Kennedy of The Shaws: Is green good?

Ravi Naik: Green is good; red is bad. As somebody who is colour blind, I have to—

Peter Swallow: Normally red is good, but in this context, it is bad.

Ravi Naik: Well, I am a Spurs fan, so maybe not. That is for the record as well. So you can see we have done an analysis. What laws are available, and how realistic would they be for somebody to pass against? This is quite a useful tool to think about not just how you regulate but what you are regulating against. What is the technology likely to be used for in a public or private context? I found this a really helpful exercise for ourselves to start to think about what laws there are and how regulators will approach these issues. I can share this report and our legal study.

A good thing that also came from this is that the Ada Lovelace Institute put together some recommendations of what the laws are, and I would like to point to just maybe three or four of them. Again, I can share this. The first is very helpful and is this idea, which Baroness Kennedy mentioned, of some form of regulator—the idea of an AI ombudsman that had the role of an actual ombudsman and could take complaints, a bit like the Financial Ombudsman, which has real teeth and enforcement powers. Would an AI ombudsman work?

Secondly, Professor Leslie talked about these five AI principles that came from the White Paper. Obviously, in the King’s Speech last July, there was an indication that there would be some sort of AI regulation so the White Paper might be outdated. But, as Professor Leslie said, those five principles are meant to go through horizontally against different regulators so that existing regulators take some bits of AI. Where there are domains where there is no specific regulator, how is that meant to work? And where there are diffuse regulations across regulators, how is that meant to work? One example we might be able to draw from is the EU AI Act, where there is a statutory basis for regulators to work together. That is quite a simple, low-hanging fruit idea that maybe we could implement.

The third one I would like to talk about, which we have already discussed, is powers for regulators to have ex ante, developer-focused, regulatory capacity—the idea that you think about how this technology may harm and affect human rights before it is deployed.

The final one—there are actually 18 altogether but I am giving you four—is worth pointing to for today’s purposes: a mandatory reporting requirement for developers of foundational models that are either operating in or selling to the UK.

Baroness Kennedy of The Shaws: Did you say a mandatory reporting requirement?

Ravi Naik: It is a mandatory reporting requirement on how their systems work. Which regulator is an interesting question. Would it be this diffuse network horizontally? Would there be some sort of specific regulator or would one regulator take that role on? For example, we saw Ofcom taking on powers under the Online Safety Act 2023. Could you just give an existing regulator those types of powers, but with some sort of mandatory reporting requirement as ongoing audit requirements with specific expertise and resources to be able to understand the technology.

One thing I am very conscious of is that we talked a lot about models, transporters, layers and all sorts of other things. I did not know a lot about this until I properly investigated; I spent hours of my life reading about how these models have developed and what they are. That is why I talked right at the start about this need for a glossary, because actually, when you break down a lot of what AI is, it is a bunch of different systems being put together and layered on top of each other. When you start to understand its component parts then you can start to understand how and what to regulate.

Peter Swallow: Professor Naik, can I just ask you the question that one is always asked when one starts talking about regulation? How can regulation be proportionate to not prevent innovation and economic growth but protect human rights?

Ravi Naik: That is a fantastic question and a real issue that becomes more of a distraction than it ought to be. Innovation can be supported by regulation because it can be a guardrail as to how technology can be appropriately used. It can allow for British companies to really take the lead and say, “We will create, develop and support responsible technology”. Something we have been talking about a lot here is regulation for the sake of having guardrails, but also regulation to ensure people are properly educated about these things and regulators themselves have the right expertise, skills and resources to be able to ensure that technology is deployed responsibly.

As Apple says, what you will find is that privacy is part of our product, and actually, if you look at the timing when Apple started saying that, it was around a lot of backlash against some tech companies that maybe did not take privacy as seriously as Apple did, and Apple saw that as a market cap opportunity. If the regulation is sophisticated, it can be used to support innovation with a view to growth, and innovation is part of that as well.

Professor David Leslie: Can I quickly come in on this? My Turing team basically led the writing of the Human Rights, Democracy, and Rule of Law Risk and Impact Assessment for the Council of Europe. I just want to quickly address this question about calibration and how we integrate a risk-based approach with a rights-based approach. If you step back and look at the knowledgeable ways that one can address that very tricky relationship, one starts from a position on the risk side where you are assessing the risks of the potential uses of the technology to human rights.

When we think of risks to human rights, there is a whole human rights due diligence machinery that outlines and understands risk. Risk involves several variables: the scale of the risk—the gravity or seriousness—the scope of the risk—how many people might be affected, for how long a time, and whether there are marginalised populations especially exposed to it—remediability, so the idea of whether those who are impacted can be restored given the violation of the right, and finally probability, the likelihood that the risk will happen. So you have scale, scope, remediability and probability.

The combination of those variables will help to calibrate proportionality. Of course, the calculus of calibration is something that needs to be part of a public policy conversation of how much we would weigh the seriousness of the potential risk against its scope, the severity or probability of the risk. That is the conventional way to look at risk from a human rights perspective; you need to have some form of calibration around the combination of those very basic variables. On the rights-based side, you have to start with an accepted set of human rights concerns based upon which you will then assess the impacts of the system. That is also how you calibrate the risk: you take the set of rights that you are going to consider now.

In the HUDERIA—the Human Rights, Democracy, and Rule of Law Risk and Impact Assessment from the Council of Europe—there are 17 areas of concern, which actually talks to Baroness Lawrence’s point. You have physical and mental integrity and human dignity; physical liberty and security and the freedom of human movement; justice, so access to remedy and due process; privacy and data protection; equality and non-discrimination; freedom of thought, conscious religion and belief; freedom of opinions, expression and information; freedom of assembly and association; property rights including ownership of physical goods and intellectual property; issues of equitable access to education, which is also considered a human right; rights around art, science, cultures, and language, so access to cultural resources and benefits of scientific progress; freedom of scientific research and scientific creative activity; labour and employment rights; health and healthcare; social protection rights; rights of children; rights that involve the environment or the integrity of the ecosystem; and then democracy and the rule of law. Those are the 17 areas of human rights that are part of the Human Rights, Democracy, and Rule of Law Risk and Impact Assessment, which forms the basis of the metric of assessment from that Council of Europe point of view.

The Chair: That is very helpful in terms of the scaffold that we are going to have to explore during this inquiry. Before we leave the issue of regulation, we have a short, sharp question from Lady Kennedy about gaps, and then we are going to hear from Juliet Campbell, Member of Parliament about bias.

Q7 Baroness Kennedy of The Shaws: How adequate is the current legal framework in the UK and where do you see gaps?

Professor David Leslie: I will start on this one. Recently, there was a Member’s Bill introduced by Lord Holmes that called for an AI authority. That Bill was flagging up some areas of concern that we should think about, and I would recommend consideration of what is in that Bill for the committee. What I would say is that first and foremost for me, the low-hanging fruits in terms of gaps are just simply transparency and accountability. We call these elements of governance the basic governance principles—end-to-end transparency—meaning that there is a deliberate way to document and evidence, a thorough process has taken place to ensure the responsibility and trustworthiness of the design, development and deployment, which we think of as process transparency, and organisational transparency in terms of supply chains and other organisational practices that have gone into the conceptualisation, scoping and then either procurement or design of the system. When we think specifically about transparency in AI, it is also the explainability or interpretability of the system so that you can actually account for the underlying rationale behind the output that the system produces.

On the accountability side, we talk about these end-to-end mechanisms of traceability and auditability in the system so that there is a comprehensive chain across the workflow where one can point to the decisions that have been made so that those who have made significant decisions can be held accountable and you do not have gaps across the value chain and the socio-technical life cycle. What I would see as an immediately addressable gap is that element of transparency and accountability, which is not technologically neutral in the sense that nothing is technologically neutral, but the transparency and accountability requirements do not speak to a specific modality of AI. They are just requirements on the practices of designing technology, and that makes the construction of legal interventions much more viable because you are not regulating the technology; you are saying, “You need to be accountable and transparent in your practices of acquiring or designing the technology”.

Ravi Naik: Can I maybe just quickly jump in? I might disagree with Professor Leslie there, which might be unusual given what we are talking about. Accountability is a very difficult challenge; it is not low-hanging fruit. Trying to take a matter to a regulator and expecting it to take action is hard. Regulators have a hard job trying to regulate this technology and it is difficult to expect accountability to just suddenly appear in that sense. Likewise, taking a matter to court is fraught with all sorts of problems. I can go into this in more detail, but I do not want to take too much time. Likewise, transparency is a much more nuanced concept because you have to think about who the transparency is for, what purpose, over what aspect, and different audiences; the public have a different expectation of transparency than a regulator or a committee.

The Chair: We would be very interested in some practicalities that you have just identified, and if you felt able to write to us afterwards perhaps fleshing that out, we would always welcome that. The more information for us, the better.

Ravi Naik: Of course.

Q8 Juliet Campbell: Some of this has been touched on earlier, but I just wanted to ask Professor Naik, given the quantity of data that is used for training AI models, to what extent do you think it is feasible for biased data to be avoided during the training?

Ravi Naik: It is a fantastic question and goes back to that point I made earlier about bias having different meaning depending who you talk to. I am not a data scientist, and it might be that Professor Leslie is better placed to answer this. But this idea of a system being only be as good as the data that goes into it is an interesting one. We all know of examples of systems producing what we would all consider to be discriminatory outputs as a result of their training data, and the need to put bias into systems to correct for the bias that comes from the training data. I can talk a lot about some work we have done when it comes to this accountability point, but Professor Leslie might be better placed to talk about the nuts and bolts of what is happening under the hood.

Juliet Campbell: If he could, and then I just have a couple of other ones.

Professor David Leslie: Again, we have to make sufficient differentiations between the different systems that are being designed. For instance, with these very large-scale generative AI systems, they are trained on such a large hyper-scale of data that actually going through and effectively addressing bias issues within the data becomes almost impossible.

Juliet Campbell: So you cannot avoid it?

Professor David Leslie: We can attempt to mitigate it in one sense or another, but there will be bias baked into that data. The real hard problem here is also issues like stereotyping. When we think of how stereotyping might arise in a very large dataset for generative AI, you cannot point to one piece of data as the source of a stereotype. A stereotype happens in what we call the latent representation space of a dataset; that is the inferential space of the data. Stereotypes emerge in that emergent way, in a sense presenting an impossibility to mitigate it. If you have a large dataset with stereotypes in it and it is the same source, the more data you put into that, the deeper entrenched the stereotypes you will find, and there are studies done on that. These are really difficult problems.

In terms of the smaller, better-curated datasets, there are bias mitigation mechanisms. I will just recommend a document that we have produced at the Turing called AI Fairness in Practice. It is part of the AI Ethics and Governance in Practice series, in which we go through ways that you can not necessarily eliminate but can mitigate bias in datasets.

Juliet Campbell: That covers a little of the next part of my question: should AI systems be required to undergo fairness audits before deployment? You are saying that yes, they should.

Professor David Leslie: Again, in this AI Fairness in Practice, there are eight textbooks as part of this AI ethics and governance series. I would recommend them. It took about six years to develop them.

When we talk about this, it will also speak to some definitional questions about what fairness and bias are. From a governance perspective, when we think of bias mitigation and fairness in the context of using AI in the UK—for instance, in the public sector—we are thinking about equality law and how direct and indirect discrimination and discriminatory harassment crop up in the design and use of the technologies. If we are specifically talking about public sector usage, we are also going to think about the more positive responsibilities, such as the equality duty, and how we step back and think about the design decisions that might impact the public sector equality duty.

All that said, there are various ways we can typologise the moments of bias in the decision-making process that need to be mitigated. I will just use one example of that. In AI Fairness in Practice, we have a four-quadrant mapping of all the different biases that emerge in the technologies, but one example in the design of the technology is aggregation bias. Oftentimes, you will have a predictive system that is built, but when they test it to validate it they do not test it based upon populational subgroups, so there is a bias in saying, “You’ve got this performance metric, there is an 80% accuracy”—just as an example—“of this system”.

But actually, if you were to disaggregate it and see what the accuracy is for this demographic group and that demographic group, it might be 98% accurate for one demographic group and 40% accurate for another. So here is an example of a bias that might emerge, that if you do not think about that in an ex ante and deliberate way, you will just not catch it. So there is a methodology of bias mitigation that needs to be undertaken.

The Chair: I know that my colleague, Mr Afzal Khan, is going to take us further into that question, but I was recently struck listening to celebrated human rights commentator, our colleague Baroness Shami Chakrabarti, saying, “If you put rubbish in, you’ll get rubbish out”. It was rather pithily put and very vivid.

Q9 Afzal Khan: Can I start by thanking both professors for being here and answering these questions? It is a fascinating and complex area. The question is how can regulators mitigate the risk that biased data is the source of discriminatory outcomes?

Ravi Naik: It is a very good and difficult question because regulators have different mandates, and you can have very narrow, sectoral-specific regulators. For example, let us take the Gambling Commission, which looks at gambling, but within online gambling there is a lot of technology being deployed. That regulator has quite a narrow focus, but has to look at technology by dint of what it is regulating. Does that regulator have the expertise and skillset to understand algorithms, algorithmic fairness and bias from all perspectives? I do not know what the answer is, but I would venture to say they might need some help, because you cannot expect a regulator to have this inbuilt knowledge of what these systems are, how these models operate, and what the real risks are compared to these overzealous, hyped risks. All regulators might need a bit of support on this because it is very difficult; that is one of the Ada Lovelace recommendations.

Conversely, take the Financial Ombudsman, which has quite strong powers, can understand a lot about how decisions were reached and really look under the hood, and probably has a lot of expertise thinking about these things. It provides a free-of-charge resolution, you have a good back and forth with it, and it is a good process. You could take the example of something going wrong with a biometric mortgage assessment using AI, and the ombudsman could potentially give you a good outcome. It really depends on the scale of the regulator, what its powers are, and then there has to be this thing of the sea of regulatory knowledge, expertise and skills—being able to lift—because all regulators are going to have AI issues coming across their plate.

Professor David Leslie: I would add one thing to this, which is there needs to be a distinctively whole-of-government, whole-of-regulatory-powers approach. That is to say you have certain regulators who have really sophisticated expertise—for instance, the Equality and Human Rights Commission—when it comes to thinking about some nuances of mitigating bias, especially in light of potentials for direct and indirect discrimination in data and design.

We really need to pay attention to building common capacity across the regulators, whether that means having a common capacity approach that is led by, for example, the Digital Regulation Cooperation Forum—a cross-regulator initiative that is thinking more horizontally about this stuff—or just a distinctive organising pattern that brings the specific expertise into play. For instance, in the domain of health and social care, there is a very rich knowledge base on thinking about bias and discrimination.

In the health use case, it is not just a high-impact area but an extremely rich environment where there have just been lots of historical examinations of the way that discrimination and bias crop up across the cascading elements of health technology. We led the research behind the Equity in Medical Devices: Independent Review in relation to health inequity in AI-enabled medical devices. In that research, we saw that actually there is a lot of knowledge that could be drawn on by other domains in order to have a better approach to bias mitigation or addressing the risks of bias. Generally, we need a more whole-of-government approach to this.

Ravi Naik: Can I just add one final thing on this? There is also a really important issue about accountability after the event; it is not just before ensuring the training data. A recent example is that the Dutch had a welfare automated assessment process, and the data protection regulator uncovered that there were a lot of flaws in the system and it had caused all sorts of problems in people being accused of fraud. Because there was accountability after the event, it was really important to ensure that system was corrected.

Afzal Khan: Let me ask you two supplementary questions. I will ask them and then you can decide who answers, perhaps one each. First, is it possible for AI to be used as a tool to prevent discrimination and do you have any examples? The second supplementary is: what particular groups of people might be most at risk from harm due to the use of biased data, or are we all at risk of harm?

Professor David Leslie: I will take the first one. The answer is yes, but with a qualification. The scales of analysis, speed and data production have meant that one should always think of AI as both an object and a tool of regulation and governance. For instance, one can use algorithmic techniques to address and filter biases or elements of toxicity in datasets. We also know that sometimes if you use AI in a tool in that way, for instance, it can act in a biased way itself. Just to give an example, if one is trying to filter out harmful language in a dataset, it has been found that it will take the language use of marginalised groups, for instance, and just filter that out in an indiscriminatory way.

So you always need to have a careful human knowledge and presence in that process to ensure that that type of thing is not happening. What that means is AI can be a very supportive tool, and it needs to be there because we need to address speed and scale in a world in which there is so much more automation and the use of AI. At the same time, we cannot become solutionist about it. AI is not the deus ex machina that is going to drop down and debias everything; it will just not happen that way. We need to manage the presence of AI as an instrument of our use that is ultimately subject to our own standards rather than a catch-all solution.

Ravi Naik: Very quickly, on the second question, it depends on the use case. You can imagine a specific minority of people being affected more in an immigration context use of AI than, say, advertising on Facebook, which might be completely different. I can pick that up in more detail in writing if that would be helpful.

The Chair: Again, that would be very helpful to the committee. We are going to go back to Baroness Lawrence. After that, we will be hearing from my colleague, Lord Dholakia.

Q10 Baroness Lawrence of Clarendon: How can AI be used for surveillance of individuals or groups and what impact might the surveillance have on individual freedom of expression and assembly?

Ravi Naik: Fantastic question. There are numerous different ways. There is a growing tendency on law enforcement agencies and security services to rely on technology. A starting point here is: when we say AI, what do we mean? If we take automation in the broader sense, we already have examples of discriminatory practices in law enforcement. For example, the Gangs Matrix was a matter on which I worked. This was a very rudimentary system, but it ranked individuals based—effectively on their ethnicity—as their propensity to be a member of a gang. That was an example of automation being inherently problematic just in the way it was built. You also have facial recognition technology and the way that is being used in conjunction with existing datasets to make predictions of people and how they might behave.

You see more—sophisticated is probably the wrong word—developed technology being used for predictive tools. Particularly in America, we see such technology being used in the criminal justice system, but we see the same thing being used here as well. For example, the COMPAS system—for deciding whether people get bail—is based on automation and the basis for that automation is a dataset that has biases within it. So there are numerous areas; we will be here for a long time explaining them.

Baroness Lawrence of Clarendon: Following on from that, I would like to ask Professor Leslie what safeguard is needed to protect people from surveillance tools being used maliciously or dangerously?

Professor David Leslie: To answer this question, there are certain trajectories. First and foremost, in accordance with the importance of public consent and democratic governance in the actual production of a technology, there simply needs to be more public involvement in understanding when and where these systems are being designed and used, especially in the public sector context. Communities have to actually have a say in what they are comfortable with in terms of potential trade-offs between public security, potential positive uses of some form of surveillance technology, and the costs that can be the chilling effects on freedom of expression, freedom of assembly and one’s assumed right to anonymity in the public. We have all these characteristics that we actually need to be able to have freedom of movement in the world. The public should be involved to weigh and consider the justification of using that type of technology in a given context.

That being said, there also needs to be much more transparency about use; the public should be aware when these technologies are being used to measure or observe them. If you have public cameras that are using AI, there has to be some mechanism that allows for public recognition that there is the use of a system like this out there in the world, and mechanisms of recourse if one is involved in a situation where there is actually harm from that.

I will also quickly just extend how we think about surveillance a little more. There are also the elements of consumer surveillance that we really need to be aware of, for instance, when we are interacting with language models. Already in the works now are ways in which our interactions with the language models of the world will produce a lot of data in real time that can then be used to not only curate our experience but capture our intentionality and second-order desires. We have lived through what is considered to be the attention economy in the past few years; we are kept on the technologies in virtue of the stream of information that is hyper-personalised to us.

As we move into a generative AI era of consumer interaction, these systems will not just try to capture our attention but our intentions. So if you are interacting with a chatbot and expressing a desire that you like a type of movie, in that moment that you express that desire, there is this moment of consumer surveillance. A week later, that system might feed back to you some recommendations on movies that you should see that it is getting paid to present by advertisers. That is just an innocuous example, but you can think of less innocuous examples of that.

Baroness Lawrence of Clarendon: But the consumer would not know that this surveillance has happened.

Ravi Naik: That is right.

Professor David Leslie: We need to make sure that there is a making explicit of these particular interactions. One element that is really important is that you understand how your data is being used, if it is being collected, and for what purposes. Oftentimes, making explicit is not part of the interaction with the platforms, which is a big concern.

The Chair: We would be interested to hear more about part of what Lady Lawrence has just asked you concerning the impact on individuals or groups. You said something about domestic application, maybe using cameras. Earlier this week, for instance, the Canadian Government took action against Hikvision. In this country, Hikvision cameras have been banned from sensitive sites by the Government.

But of course, we know that in places like Xinjiang, whole groups of people—Uyghur Muslims—have been under camera surveillance. We would be very interested to know how authoritarian, autocratic and totalitarian regimes are able to apply AI in addition to that technology that is already there. Perhaps you could reflect on that. I know that this issue domestically is a point that Lord Dholakia wants to ask you about, and the question of privacy that you raised earlier on.

Q11 Lord Dholakia: How do the rights of the AI systems impact on the right to privacy? Three areas come to mind very quickly: facial recognition, predictive analytics and surveillance. I took part in a demonstration that the Metropolitan Police had set up in the House of Lords, and I was not very impressed at all.

The Chair: Who wants to go first on that?

Professor David Leslie: I can go first and just say that we live in an increasingly cyber-physical environment. There are multiplying points of measurement and observation. It could be cameras for facial recognition or the way that our phones are measuring our movements. We live in a world in which there are so many more sensors and actuators around us that managing our freedom of movement, assembly and capacity to interact without living under the watchful eye of data extraction and collection is becoming—or should become—an increasing priority.

What we first and foremost need to ensure—maybe I will repeat myself a little here—is that the public physical space is a space where we take part and share in the moral, political and social life of our community. In no way should that space be compromised by the effects of surveillance technologies. We—meaning the impacted communities and affected groups—should always be fully aware of what is happening in that space and have agency in determining whether the technologies that are out there should or can be measuring us in those spaces. That calls for some reassertion of democratic governance and public consent in the way these technologies are being designed and used, but it also demands that we start looking at our everyday life a bit differently.

A good, extreme example of this is the metaverse. I will just quickly say that we have not talked a lot about the metaverse recently in the public imagination, but if you can imagine a space of interaction where there are blank cheques for surveillance technology and measurement. Everything that you do in a digital environment like the metaverse is measured in one way or another. From thinking about that extreme case, let us start to think about the way that we move around in physical space. At the end of the day, the deep demand and desire for training data means that we will not be able to manage the transformative effects of the technology unless we actively and proactively take a role in managing all the different flows of data extraction and collection.

Ravi Naik: Can I maybe add a balancing comment to that? You talked about the demand of the public to change how they interact or think about their built environment. I am not sure that is right because I do not think it is a realistic demand for people to start to modify their entire behaviour in their day-to-day life. Life is hard enough for most people. Really, what you have to think about is what people can realistically do day to day, and if this has become a reality, where does the burden shift and fall? I do not think it is a fair burden to expect every single citizen to try to modify their behaviour or understand what is happening around them. For example, you can take going shopping where there is now facial recognition by private actors. Are you expecting people to understand what is happening with that technology, how to make objection requests and so on? We have to think about realism and what is realistic, particularly from a human rights perspective, and that is maybe where we have to start to think slightly more at a macro level about what the governance is and what we need to govern.

Professor David Leslie: Just to clarify, I was not saying that individuals need to be fully aware all the time. What needs to happen is that the public need to be involved in these decisions about using, designing and deploying systems and data collection so that there can be structural public determinations of what is and is not acceptable. In other words, I do not want to make the public feel responsible for being aware at all points of how they are being surveilled, though that is an important element. We need to empower members of the public to have a sense that the space that they are moving around in is a space of freedom, it is their collective space, and that they should not be involved in the conceptualisation and governance of very impactful technologies. It is not that there needs to be a constant and mass awareness of this at all points. We can shape innovation ecosystems so that there is more rather than less empowerment of members of the public.

The Chair: Of course, it is worth reminding people that this is a parliamentary hearing. Parliament is here to represent the public, and the public are entitled—if they wish—to make written representations. We have made a call for evidence for this inquiry, and I hope they will respond in the way that you have both been saying and give us their views about both the opportunities and the dangers that are presented. My colleague Dr Swallow is going to come back and ask you a supplementary question, and then we will hear from Tom Gordon, Member of Parliament.

Peter Swallow: Just to touch on that very important point, I want to invite you to comment on the Government’s commitment to publishing their algorithms and the algorithms used by government departments. I appreciate that not every algorithm is AI, but nevertheless it is an important transparency measure, is it not, so that the public are certainly at least aware of how the public sector is proposing to use data to engage in some technologies.

Professor David Leslie: That element of transparency that has now been codified in the procedures of government is a great step forward because having a proper register and making that publicly available is a crucial step forward in ensuring that there is sufficient awareness. Over the past few years, I know there have been real struggles at the local authority level, for instance, about understanding where AI is being used as predictive risk modelling in children’s social care, or how AI is being used with the local police force, et cetera. There have been real barriers to having proper transparency on that. As we move forward, there needs to be even more deepening of that relationship of transparency between public sector bodies and the impacted communities.

Ravi Naik: I would second that.

Q12 Tom Gordon: I know you mentioned this earlier in responses to some other questions when we were talking about how data might be used and such. I just wondered if you would be able to elaborate and expand upon what sorts of data organisations may have about individuals, how they process it, and what potential risks there are to the individual as a result of that data being held and processed by organisations.

Ravi Naik: Do you mean from a public authority perspective? Which organisational perspective?

Tom Gordon: More to the rights of the individual. What risks does it pose to them if these organisations have it? Obviously, we have GDPR and much data protection. I am less worried about the impact on the business or organisation; it is more about the rights of the individual and what potential issues might arise from that.

Ravi Naik: I am taking the question as meaning the organisation, for example, the tech company.

Tom Gordon: Yes.

Ravi Naik: It is a fantastic question. In theory, we have a very strong data protection framework, as you say, the GDPR, or the UK GDPR as it is now, and we have a series of ancillary rights. In theory, they provide a really strong bulwark against these AI systems if your personal data is part of that system. If there is inaccurate information about you or you just want to object to the use of your data, in theory that all exists and is powerful and available. I emphasise the word “theory” because the reality of trying to enforce these rights is quite difficult for two main reasons.

One is if you try to object or have your information erased from a system, what level is that erasure occurring at? Is it just the output that the system gives back to you? If somebody says, “Who is Ravi Naik?” and it says I am an Arsenal fan but I really do not want it to say that and ask it to be corrected, at what level is the correction happening? There is a really interesting question. Are you affecting the intrinsic model or just the output and the way it talks back to you? There is a layer of technology called system prompts, and there is a really interesting dynamic about what a system prompt is, how it works and whether the system prompt is effectuating those rights. I can maybe follow up in writing on system prompts.

The second problem—which I have talked about quite a lot—is the reality of trying to enforce and get accountability. The ICO is a stretched organisation. I do not think we could expect the ICO to enforce every single complaint to its natural logical conclusion because that would just be overburdening one regulator. Taking court action is likewise going to be problematic, costly and take a lot of time. The ICO could maybe do more—as could every regulator—but there is this question of: how do we ensure that people can really hold up to those rights? Unless you can stand by rights, they are hollow. I hope that answers where you were going with that question.

Tom Gordon: It does. Professor Leslie, is there anything that you want to add?

Professor David Leslie: I agree with all that. We also need to consider issues of inference and the way in which our data—as it is handled in the ecosystem—will also involve our grouping and inferences that can be made about us vis-à-vis other people. There are real consequences to that fact that are not necessarily picked up in an individual if you are just thinking about an individual and their data rights. There is a layer of consideration of the way that inferences group us together that also needs to be involved in those considerations.

Tom Gordon: We have talked about the data and how that might impact on the rights of the individual. Taking it one step further, to what extent do we think there are currently adequate protections against the inappropriate use of AI using that data?

Ravi Naik: Again, you fall back on the data protection framework and the intrinsic positives and negatives there. The problems with misuse of that data by a third actor are, first, identifying where it has occurred, and secondly, accountability. We have done a lot of work for individuals who are victims of revenge pornography or manipulation of their image. Often, the reality of the law hitting the road is that these people are offshore, or they hide it in a company and that company is somewhere else, and it is almost impossible to get accountability that way. Almost the act of instigating illegal action causes that organisation to double down and maybe put out more of this same horrible information. Maybe there are changes that need to be brought to the law about jurisdiction and extraterritoriality of some issues because a lot of this technology is not located just in this country; it is services operated across the world. How we deal with extraterritoriality of these problems is why I am sitting on this side of the table rather than there.

Professor David Leslie: Just to say that there are significant enforcement gaps, which we all know right now. For instance, we did a study on the impacts of generative AI on the creative workforce and one element that arose there was the fact that voice data—biometric data—was just extracted from certain people and then used to generate synthetic voices. There are real concerns about the fact that it is a chaotic space where really these larger companies will just help themselves to our data, knowing that there is such a high level of knowledge and resource asymmetry between consumers, regulators and larger corporations that they can basically act in many ways with impunity. There is a real necessity of thinking at that structural level; rather than focusing on what an individual can do to have recourse in these situations, to think about what structural conditions can be put in place to ensure that you will not have illegal appropriation of biometric data and good enforcement of that type of thing.

Tom Gordon: There was an interesting example, actually. A colleague of mine, Max Wilkinson—the Lib Dem MP for Cheltenham—was alleged to have chuntered some words that I shall not repeat about one of the Reform MPs. It turned out that had been the use of AI to generate text that he did not say, actually using the live feed from the Parliament Chamber. So it is quite worrying for people in the public eye but also for rights to private, individual life.

Ravi Naik: Can I just add one further thing on this? We have focused a lot on the misuse of AI or people’s likeness. There is another issue that we are dealing with a lot in our work, which is taking individuals' actual likeness—purportedly with contractual consent—in the creative industry. You have individuals in the creative industry having their likeness, voices and musicians taken, and so on, and then that information being used to create digital likenesses of themselves. That is a real problem, and where this becomes really problematic is when we try to enforce the law against those companies, which are often not big companies, and it becomes a question of just contractual payouts.

Tom Gordon: But there is actually—

The Chair: I am anxious we should move on to the questions of redress, Mr Gordon, which indeed I know that Lord Sewell would like to ask about.

Ravi Naik: I am happy to follow up.

The Chair: If there is more, we can come back.

Q13 Lord Sewell of Sanderstead: I just wanted to ask: should individuals have the right to opt out of AI-driven data collection and profiling?

Ravi Naik: Yes, they should. I would say a few different things. You were talking about opting in and opting out. People were a bit late; the horse is already out of the barn, as it were, because a lot of these models are already created and out there. But when you think about the training stage, people should be asked if they want their information used in a particular way and for particular purposes. Once you have already had that training, people should have the right to very easily—probably by technological means—opt out, object, and have their information stripped from the system. Whether that is stripped on the basis of it being on the system front level, I do not think that is enough; that information should be stripped from the underlying model.

Lord Sewell of Sanderstead: I am talking about a technology that wants to reinforce your own prejudices. You go on your YouTube and see the algorithms, which will pop up centre left, centre right or whatever, and you really do not have a say in that. It just tends to reinforce, and there are no options really; it just does it subtly.

Ravi Naik: Two things on that. One, YouTube has actually turned off its recommendation systems, so you can actually now just have YouTube where you just watch the video. It is quite an interesting development.

Lord Sewell of Sanderstead: I wonder how many people do that.

Ravi Naik: It is just an account setting, which is interesting. It has chosen to make that decision. It might be because it has a captive audience. Everyone uses YouTube; it has market dominance and can do things like that.

The second thing is that we have rights. I do not think YouTube did this just out of the goodness of its own heart, and I do not know if I can get into the details of the cases, but there are legal mechanisms to stop recommendation systems in that way. I can maybe talk about it privately, but I do not think it is a free-for-all. This is an accepted reality, but the law actually prohibits that type of conduct.

Professor David Leslie: I will add one thing to this. In thinking ahead, we now need to consider emerging risks in this space. One element that I would recommend the committee really considers deeply over the course of the inquiry is the emergence of agentic AI, which is to say the use of these systems out there in the world as acting agents. I will give a relevant example here. Right now, there is a technological capacity to weaponise chatbots that are hyper-personalised to individuals. You will be interacting on the internet in your normal modality, and suddenly, in social media or some other platform, you will be engaged in chat by what would appear to be a human agent, someone who has a like interest to you, maybe likes what you have said, and then makes a comment and starts direct messaging with you.

What we know is that with agentic AI, there is a very low-cost way of someone with, say, interest in swaying you in one direction or another to use a technology that—on the basis of your behaviour—will empathise with you and create conditions where your own leanings are reinforced, or you are persuaded to have different leanings based upon what those who are designing the agentic chatbot would want. This kind of weaponisation of hyper-personalised systems is a real threat to some basic elements of clean public communication that has integrity and that we might not otherwise be aware of. It is something that is coming, if not already very present.

Q14 The Chair: I know that Mr Sobel wants to ask you about accountability where things go wrong, but if I can just pre-empt that by asking about redress. If you could just talk us through what redress would be available in the sort of cases that Mr Gordon and others have mentioned during the course of these proceedings. What can you do when things go wrong, and is there more that we should be recommending as a committee that could be done at a practical level?

Ravi Naik: This is my day to day, so I am happy to take this. We can split it into public and private actors, and that is quite a useful split because it goes to the remedies that are available. For public actors, obviously, you have judicial review processes, which is quite helpful because you have public law principles, and they are quite helpful when paired against the use of technology. Public law is about decisions; how did you get to a decision? You can start to unpick the technology and understand if the decision was fair, for example.

With private actors, things get a bit muddier because you are reliant on the civil procedure. There is one area of human rights law in the Data Protection Act 2018 that gives a very good remedy: you can get non-monetary relief through what is called a compliance order. There are numerous cases that we have brought. There was one recently of Tanya O’Carroll v Meta. All the information is public. It is not an ongoing case, but we sought a compliance order there. That was a really powerful example, almost going to Lord Sewell’s point about what you do if you have these recommendation systems. You do not actually want money. It is not about financial redress; it is just stopping a system from operating in a particular way. So the data protection regime provides that. Outside the data protection regime, the Human Rights Act 1998 obviously provides for non-monetary relief, but it only applies to public actors, so maybe that is an example of where the development or evolution of legal protection could really help individuals.

I would say the final layer is regulatory oversight, and where you have an ombudsman, such as the Financial Ombudsman, which I keep going back to, and a cost-free, properly adversarial approach to redress, that is really helpful. But the Financial Ombudsman is an outlier, and it would be interesting to see if we could maybe improve that mechanism in the digital space.

The Chair: That is a really interesting, helpful reply. Professor Leslie, do you want to add to that?

Professor David Leslie: I would just say that in a constructive, forward-looking way, the work that the Council of Europe is doing to think about how one can have an equitable process of setting up the procedural rights for actionable recourse and effective remedy involves—in the design of the system—setting up a process where issues of remediability can be explored with potentially impacted people. That means you can start to set the right type of mitigations in place, and plan and have accountability in that as a part of the actual governance of the technology. It is a really essential feature of establishing a rights-based approach to the design of these technologies: that you actually have structured protocols for establishing paths to remedy.

The Chair: Forgive me if I have misunderstood, but I think you have both talked a lot about the danger of there being a lack of transparency and how that can jeopardise human rights. If that is so, how do we police that and provide remedies for that kind of breach of privacy?

Ravi Naik: It goes back to my slight disagreement with Professor Leslie about what we mean by transparency and who the transparency is for. We have cases where we have actually sued for transparency. I have deliberately tried not to talk about financial compensation because I do not think that is what really causes systemic change. But seeking orders for transparency over what the information relates to—for example, the code that led to a decision—might be the appropriate remedy for a particular individual because then you could ask about remedies over that code.

The Chair: You do not have a copy of what a code like that might look like, do you?

Ravi Naik: Yes, we have examples from our work that I would be happy to share. I can follow up in writing as to some cases we have in a higher level so you can understand this idea of transparency—almost in itself—being redress. Because once you understand what is going on under the hood within the system, if you have that transparency, maybe that gives you the answer. That is all we wanted all along.

The Chair: I know that there are some cases that we cannot talk about because they are sub judice, but hopefully, as they are settled, you will be able to share with us some insights, what we might learn from those cases, and therefore how the law might be changed to deal with future occurrences. Let me hand the floor now for the penultimate question—you will be relieved to hear on this hot and rather sweaty day—to Mr Alex Sobel.

Q15 Alex Sobel: In 2018, 2,400 individuals who worked in tech, regulatory and legislative roles signed a pledge not to support the development, manufacture, trade or use of lethal autonomous weapons. Since then, one of the signatories—Elon Musk—rose into great notoriety. I was the only parliamentarian to sign that pledge, so that is the only thing I have in common with Elon Musk. But the point is that the other thing that has risen is the use of lethal autonomous weapons in conflicts, particularly in Ukraine and Israel-Gaza, and so we are much nearer the reality of lethal autonomous weapons than we were in 2018. My question is if a lethal autonomous weapon massacres a village, who is legally responsible and accountable?

Ravi Naik: I can very quickly say I have actually written a paper for Harvard thinking about this idea of accountability and liability gaps, which I can share with you. It is very complicated. That book I referred to earlier—Robot Rules—also looks at this exact question. It is very complicated and that is a vacuum in the law that probably needs filling. Professor Leslie, I have talked a lot, so I will hand over to you.

Professor David Leslie: If I can just address the comment indirectly and say that we really need to be paying attention to tendencies towards militarisation and weaponisation of AI. We know that AI is a dual-use technology, so you can have extremely valuable public use cases for the same technology that can also have extremely lethal consequences. But beyond this, we see this mobilisation now of increasingly lethal approaches to automating, for instance, with AI-enhanced drone technologies. We are very close—if not already there—to having drone swarm technologies, which will have an increasing presence. Of course, there is international humanitarian law, but right now, again, the claims for the priority of national security have tended to create conditions of a race to advance lethal, AI-enhanced technologies.

One element that we really need to pay attention to is what we would call the race for speed. AI is increasingly being used to increase the speed of strategic and lethal decision-making on and around the battlefield. Claims for needing optimal speed have again allowed for there to be a sidestepping of viable controls over the way we think about what is actually justifiable in terms of the production of these technologies. Speed is a military advantage and it has provided a justification for what some see—in a very realistic sense—as an Oppenheimer moment for AI. There is this idea that the increased lethality of militarised AI presents a future in which fully automated killing on the battlefields will mean the victor is just the technological or those who can produce superior software. That is a very dark tunnel to walk down if we do not do something to really rethink that.

Ravi Naik: Can I maybe just shed some light into that dark tunnel that might be helpful? I do not actually know if the laws of war are the best example because they are actually quite sophisticated at apportioning liability. In that example, if it is on behalf of a state, you could imagine a chain where there is liability. The International Criminal Court has some very interesting precedents on this.

Really, what your question goes to is liability gaps for automation. You can actually imagine a more rudimentary example, for example, a contract for a lawyer doing some basic services for you in conveyancing. That is actually where most people are going to face the reality of interaction with AI and where this question starts to really hone in.

Q16 The Chair: Gentlemen, you have certainly given us an awful lot to think about. In a way, the final question would be if there were to be a government Bill on AI, what would it look like? What should be in it? You have actually covered a lot of the ground in the course of the hearing, but if you would like to just give us some headline points, that would be very helpful and a good way to conclude what has been a fascinating session. Shall we start with you, Professor Naik?

Ravi Naik: It is a big one to end on. If I could legislate in this space, I would do a few things. First, I would think about what it is that we are legislating against. I make this point quite often because it is very important, which is thinking about definitional thresholds and red lines around what the technology is and is not and how we distinguish it, maybe thinking about glossary of terms and so on.

The other thing I would really emphasise if I could legislate in a utopic way is making redress easier for people. The thing that saddens me most about my work is having to say to individuals, “You’ve got a really meritorious case, but the cost of action is just so prohibitive that even if we acted pro bono for you, it’s not our costs you have to worry about”. To me, that is a real tragedy in this space, and the lack of accountability being caused by costs and how we make redressing accountability real is the thing I would push for.

The Chair: Your last word on this, Professor Leslie?

Professor David Leslie: My last word would be that I mentioned that transparency and accountability should be very big priorities when it comes to how we are constraining the private and public sector actors in this. We need to demand end-to-end accountability and transparency, both in the design of the systems and in the systems themselves, and the organisations that are developing, procuring, and using them.

I will add to this that as part of that kind of approach, we should also have requirements for ex ante risk and impact assessment because there has been a tendency towards thinking about AI in the wild, in an ex post way: how do we control and audit it? That will never address the root causes of the issues that are causing social harm out there; there has to be a codified approach to some form of ex ante risk and impact assessment with transparency requirements.

I will just quickly finally say that there was a lot of talk the past few years about how AI is bringing us to this axial point where you can either use this technology to advance the public good in ways that really address sustainable development goals, advance human rights, address health issues, et cetera, at the same time as there is this other path where we can end up in extremely deep harms, risk and whatnot.

If that is the case—which it is; we are at a pivot point—then the law should think about how one can see the public interest implications of that much power being situated in decision-making that is largely dominated by a few private sector actors at the moment. It is a larger issue, but if it is the case that we are intending to protect the public interest and see the possible positive use cases for AI, we need to widen the lens and think about how we have the hands that are on the steering wheel steer it towards the public good rather than the negative externalities.

The Chair: Thank you, Professor Leslie. That is a very good note for us to end on because it is about steering the wheel for the public good and making sure that we get the best out of AI, not getting too fearful of it. We are not Luddites and do not want to destroy the machines, as it were; we want to harness AI for the common good, and we have to safeguard the rights of people; that is our job. This gives us a different dimension from many other inquiries there have been into AI. This is about ordinary people’s human rights.

We have been privileged to have you both with us today. I hope that you do not see this as the end of the discussion and that you will continue to engage with us as we proceed with the inquiry. As I said earlier, people themselves will be able to make their written submissions to the committee. I should emphasise that the actual opening date for that is literally just before recess. We will publish it on our website but people can start putting their thoughts together as a result of hearing what you have said to us today. We will also be putting our thoughts together in thinking about what Government should be doing and what new legislation and unfolding policy should look like. We have been blessed with two very eminent and distinguished contributors as witnesses today. Thank you both very much indeed. With those words, I close this hearing.