AIG0002

Written evidence submitted by Dr Tetyana Krupiy

(Lecturer at Newcastle University)

Reason for submitting the evidence: I have been conducting research on the legal and social issues arising from the use of artificial intelligence for over seven years. I am concerned by the rapid expansion of the use of artificial intelligence to make predictions about people and to make decisions about them by the public authorities given the inherent limitations of this technology.

Summary: Artificial intelligence is not a reliable tool for making predictions about peoples’ behaviour. People cannot exercise adequate oversight over artificial intelligence technology. The government should enact a law prohibiting the use of artificial intelligence to produce predictions about people and to make decisions about them. This ban should cover partial and full reliance on artificial intelligence as part of the decision-making process. Moreover, the government authorities should not use artificial intelligence without people giving informed consent to such employment of technology. The document has 4 pages.

Risks and opportunities of artificial intelligence adoption in government

1. Artificial intelligence is suitable for use in contexts by public authorities where there are fixed parameters which determine possibilities. Although there can be different pathways to achieving the desired outcome, there are parameters determining what pathways can be pursued and what results are possible. People’s behaviour does not fall under this category.

2. Government authorities should not employ artificial intelligence to predict peoples’ behaviour and to make decisions about them. People are inherently complex and unpredictable. They make decisions and choose actions in a very complex social context. Computer scientists acknowledge that the use of artificial intelligence has innate limitations and does not permit to achieve accuracy.[1] The performance of artificial intelligence does “not correspond to lay understandings of ‘prediction.’”[2] The subjective discretionary decisions of the programmer shape what predictions the system produces about a person and what decisions it generates.[3] Given this state of affairs, the public authorities should not use artificial intelligence to make predictions and to reach decisions about people. For example, the government should not use artificial intelligence to determine educational opportunities of students.[4] It should not assess whether there is a need for intervention in regard to children using artificial intelligence; an example is reaching a decision whether to take a child into care.[5]

3. International human rights law treaties to which the UK is a party[6] place limitations on the use of artificial intelligence as part of the decision-making process. The full automation of the decision-making process using artificial intelligence is in breach of the prohibition of discrimination in the Convention on the Elimination of All Forms of Discrimination Against Women (CEDAW).[7] The prohibition of discrimination in the Convention on the Rights of Persons with Disabilities obliges states to seek the consent of the persons of disabilities before subjecting them to the partial or full automation of the decision-making process using artificial intelligence.[8] Persons with disabilities can object to the use of artificial intelligence as part of the decision-making process without the need to request reasonable accommodation.[9] Martin Scheinin showed that this requirement to obtain consent from the subjects of the decision-making prior to being able to employ artificial intelligence extends to other human rights treaties including CEDAW, International Convention on the Elimination of All Forms of Racial Discrimination[10] and International Covenant on Civil and Political Rights.[11]

4. In my opinion there is a need for legislation prohibiting partial or full reliance on the employment of artificial intelligence to make predictions about individuals and to reach decisions about them.[12] More broadly, the government authorities should not use artificial intelligence technology without the prior informed consent of the individual affected by the deployment of artificial intelligence technology.

Departmental accountability in the context of artificial intelligence: delivery, funding and implementation

5. Researcher Riika Koulu demonstrated that human beings lack the capacity to oversee the workings of complex systems, including artificial intelligence.[13] Matilda Arvidsson and Gregor Noll  believe that the use of artificial intelligence shifts decision-making to a less democratic, accessible and transparent process.[14] Using artificial intelligence to explain the basis for a decision to the subject of the decision-making does not provide meaningful information about what this person can change to achieve a positive outcome.[15] Some types of explanations can hide the fact that the decision was based on a protected characteristic, such as race.[16] Computer scientists can make arbitrary choices which change what kind of explanation the artificial intelligence generates for the decision.[17] Consequently, it is hard to achieve appropriate human oversight and accountability when public authorities use artificial intelligence as part of the decision-making process.

6. For this reason, public authorities should not use artificial intelligence to predict people’s behaviour and to issue decisions about people. They should treat partial automation of the decision-making process as posing the same degree of risk to breaching fundamental rights and social values as the full automation of the decision-making process.

Progress on strategy development and governance arrangements

7. There is a need for legislation prohibiting the use of artificial intelligence to make predictions about people and to reach decisions about them.

Data and skills issues in government

8. The evaluation of whether an artificial intelligence system produced an appropriate prediction about a person and decision requires a collaboration between experts in law, computer science and social scientists from numerous disciplines. Such disciplines can be decolonial computing, critical data studies, sociology and anthropology among others. This process of evaluating the outputs of artificial intelligence is very time consuming, laborious and expensive. This process of detecting problems with the operation of artificial intelligence is hindered by the opaqueness of these systems.[18] Samir Rawashdeh explains that “we can’t trace the system’s thought process and see why it made this decision.”[19] Given this state of affairs, I do not think that training staff members will address the problems associated with using artificial intelligence to make predictions about people and to make decisions about them.

April 2024