Istanbul Bar Association— Written evidence (NTL0028)
 

We have answered three questions that you publish on website, below:

1. Do you know of technologies being used in the application of the law? Where? By whom? For what purpose?

Over the past several years, technology has had a significant impact on the application of law. On one hand, with the rise of the internet, legal information has become much more accessible for most people who have access to the internet. On the other hand, considering the differences between the generations, now legal professionals tend to take advantage of the technology more than ever, and even to become the creator of it. But the pursuers of technology users in the application of the law are not limited to average citizens or legal professionals, but also governments and official agencies, etc.

Taking into account how technology facilitates some processes in the application of the law, to get the maximum benefit from it, every angle of law, even the dark ones, are being evaluated for the possibility to integrate law and practice. Some attempts to achieve this goal have been accomplished, some have not yet. Some examples of technologies being used in the application of the law, which are currently in use are as below:

2. How do technologies impact upon the rule of law and trust in the rule of law and its application? Your answer could refer, for example, to issues of equality. How could any negative impacts be mitigated?

The development, deployment and use of new technologies impact a wide range of the rule of law and human rights. In particular, as a new technology, the usages of AI systems below directly affect "freedom of expression; equality and non-discrimination; social and economic rights; fair trial; right to privacy; physical, psychological and moral integrity".

AI applications can be used in the media space with high efficiency, especially on social media platforms. By using AI systems, it is trying to keep people on the platforms as long as possible. Though they have created 'echo chambers' and 'filter bubbles', people spend a long time online. Moreover, to increase and facilitate this, AI systems are routinely used to select and prioritize content that keeps people on the platform. When AI systems are used irrespective of whether the content is objective, accurate, diverse, or relevant, the risk of hate speech, misinformation and disinformation quickly increases. "Deep fakes" which have become the most popular technique in recent years, are used for creating fake real people's mimics or voices. Although deep fake could be used for beneficial purposes such as data processing in autonomous vehicles to protect natural persons' face data, malicious uses on online platforms directly affect freedom of expression negatively. Since it makes it easier to spread disinformation, it causes an erosion of reaching objective and accurate information. It affects the capacity of individuals to form and develop opinions freely.

We have witnessed AI systems making biased decisions many times in the past, and unfortunately, it continues. For example, in England, AI systems were used for scoring exam results, but the algorithm placed high importance on a school's historical performance, which caused biased results for high-performing students at underperforming schools. We have faced many similar examples for a long time. It could consist of sexism, racism, ageism and other unjust discrimination and so on. Therefore, it creates a new challenge to non-discrimination and equal treatment.

The use of AI systems is increasing in workplaces. Employers mostly prefer to use AI systems to assess and predict employers' potential and performance in hiring and firing situations and monitoring and tracking. These applications include the risks of violating the right to just, safe and healthy working conditions. It causes new risks to social and economic rights. As mentioned above, the biased decision generated by AI could increase the risks of violation in the workplace. Moreover, when scoring individuals by public entities, including the social security decision, it could significantly impact individuals' lives. 

AI systems directly affect the judicial systems as well. As mentioned by the report 2018 of the CEPEJ, "it stresses the potential of AI developments to improve the predictability of the judicial process and to ensure the transparency of judges' work and the consistency of case law but also notes that such processes cannot be limited to algorithms and must take into particular account circumstances and ensure respect for fundamental rights." Indeed, it can make essential contributions to judicial systems, especially procedural economics. However, it should not be forgotten that the AI system's opacity could contradict the principle of accountability and transparency. For the efficiency of justice and its quality, accountable systems should be used, and human oversight should be ensured. (We give more details about this topic in question 18).

AI systems need big data and these mainly include personal data. For example, face and voice recognition, emotional analysis, health solutions and more, personal data could be processed. These applications could profile or categorize individuals and predict and influence their behaviors. Even if the state or private sector processes personal data under data protection regulations, the perception of monitoring and identifying continuously by AI systems could affect individuals' moral and psychological integrity. Furthermore, it can violate the essence of the right to privacy.

In line with the above explanations, it can be said that AI systems negatively affect democracy due to their negative impact on the rule of law and human rights. It requires more accountable governmental processes in the digital transformation age. Consequently, it must be taken into account all of the systems' socio-technical implications for the future legal framework on AI.

3. With regards to the use of these technologies, what costs could arise? Do the benefits outweigh these costs? Are safeguards needed to ensure that technologies cannot be used to serve purposes incompatible with a democratic society?

The use of new technologies, in particular with AI, will inevitably transform traditional approaches and methods in justice and home affairs, law enforcement, migration and the criminal justice system. This rapid and inevitable transformation should be achieved with the aim of democratic and equal society, ensuring effective exercising of fundamental rights and freedoms. The use of new technologies in certain fields of law and public administration may bring great progress and success; however, it may also lead to harmful consequences and costs[1] in many other areas such as counter-terrorism, migration, criminal justice and crime prevention. In many countries, even relatively old technologies such as CCTV and the use of data obtained via such methods are debated. Although new technologies may be useful in the fight against crime, they also have various drawbacks, especially due to the risk of turning into a large surveillance society.

Advanced algorithmic tools and AI are particularly effective at pattern recognition in huge datasets; thus, enable us to understand the correlations between incidents and phenomena that we have not been able to understand. However, when it comes to human behaviour, numerous different factors must be taken into consideration: There are always atypical circumstances and black swans. Therefore, the use of new technologies in areas such as law enforcement -where human behaviour is the subject- must be distinguished from other areas where sharper judgments and conclusions can be made, such as contract drafting. There is a requisite of human understanding, appreciation and discretion in these areas, given that AI may disregard minor but essential elements in an event. Therefore, especially in areas where human behaviour is so prominent, new technologies can and should only be an assistant.

One of the main drawbacks could be observed in the surveillance: Thanks to the new technologies, obtaining and processing data has become extremely easy compared to previous centuries, and it led people to feel that they were living in a Panopticon. The perception of cluelessness of exactly how they are being watched, poses the risk of undermining democracies. Therefore, when the use of these technologies is inevitable, transparency must be ensured (Due to the very nature of intelligence, when transparency cannot be provided, effective audit must be guaranteed).

To specify some other drawbacks; there is always the risk of bias, thus using these technologies may further reinforce existing inequalities. Use of AI systems in these specific areas may discriminate between people as “acceptable” and “not acceptable” by promoting the average qualifications of people. It is certain that the data input proceedings must be widely and effectively monitored by humans. Even when it is ensured, bias may always occur in AI systems as it may reflect the prejudices and ideologies of the persons or institutions that provide monitoring. Besides, even with cautious human control, current counter-terrorism and law enforcement activities face specific problems caused mainly by false information and the malicious use of powers by executors. It should not be forgotten that the factual root causes of social disorders need to be resolved and that new technologies are only an assistant, which in particular AI is still in its early era, thus responsibility cannot be entrusted to it with absolute trust. Determining the safeguards, it should be borne in mind that we are in a transitional period. Furthermore, considering the sensitivity of data, hacking and tricking of AI systems may extremely be detrimental, even lethal in these areas and cause irreversible damages.

Overall, the use of new technologies in specified areas poses serious risks that may directly affect people's lives. Even though it brings some achievements that humans have not been able to achieve before with traditional methods, the drawbacks (costs) are extremely grave that they cannot be disregarded. From a relatively subjective perspective, it can be argued that; when it is absolutely necessary and inevitable, these new technologies may be used only as assistant tools with constant human monitoring and effective safeguards.

5 September 2021

7