Professor Eamonn O’Neill, Director, UKRI Centre for Doctoral Training in Accountable Responsible and Transparent Artificial Intelligence (ART-AI) – Written Evidence (AIW0016)

 

ARTIFICIAL INTELLIGENCE IN WEAPONS SYSTEMS

 

This paper has been co-authored by members and partners of the UKRI Centre for Doctoral Training in Accountable, Responsible and Transparent Artificial Intelligence (ART-AI). Based at the University of Bath, ART-AI educates interdisciplinary professional experts to make the best, and safest, use of AI and to explore the opportunities, challenges and constraints presented by the diverse range of contexts for AI. We bring together PhD students, academics and external partners from a range of backgrounds in a uniquely interdisciplinary approach across computer science and AI, engineering and technology, and humanities and social sciences. Following a workshop that we hosted on AI ethics in defence in November 2022, in this paper we offer a further contribution to the debate on the ethics of using AI in weapons systems.

 

Introduction

 

  1.              The Lords’ inquiry notes that autonomous weapons systems (AWS) have been defined as systems that can select and attack a target without human intervention. For the purposes of this paper we also make the following assumptions:

 

  1.               AWS need not be lethal or have a physical impact; other forms of weapon are commonly used. Any AI enabled system that can operate independently from human users based on programmed aims, constraints and restrictions would therefore count as an AWS if it has military applications;

 

  1.               In practice, military and otherwise, the deployment of AI applications may, depending on the specific task and context, either enhance or replace human capabilities and actions.

 

  1.              We first address an underpinning issue in the procurement and deployment of AI systems for UK defence. Then, assuming we accept its premise of the ethical traditions and approach of UK military institutions and their implications for the procurement and deployment of AWS, we argue that there are situations where it could be unethical not to use AWS since they offer – or will come to offer – efficient, effective and carefully targeted performance and protection where human only systems could not.

 

Procurement and the ethics of AI in UK defence

 

  1.              As AI’s capacity to take autonomous decisions grows, questions arise about the cultural values and principles that inform decision making. ‘Military tradition’ is not just a matter of ceremony but includes embedded approaches to the ethics of war and combat. The traditions of different national armies are often ethically distinct, reflecting differing political and cultural ideologies. International approaches to what is, and is not, permissible in war – including the definition of war crimes – provide tacit recognition that the ethics of war are contentious and hard to police.

 

  1.              It matters, therefore, that the ethical assumptions built into AI design, development and deployment for UK military purposes are consistent with the ethical traditions of our own military institutions. To offer just one example, while preserving the life of one’s own troops is a priority, that objective is not to be secured through the indiscriminate taking of innumerable other lives. It is also well known that existing AI systems can incorporate unintended bias based on factors such as the characteristics of training data and the incorporation of human biases (e.g. the limitations of facial recognition technology have been widely reported) and that such biases may have cultural and ideological foundations.

 

  1.              The question therefore arises: whose assumptions, prejudices and ethical principles are incorporated (wittingly or otherwise) in AI applications which make life and death decisions – and which, in a military context, could potentially cross the line into war crimes? While human combatants can, and sometimes do, make bad decisions which breach the ethics of war, there is a danger that remitting decisions to algorithms will undermine the values that characterise our own ethics of waging war, which are themselves part of the civilisation and way of life our troops are committed to defend.

 

  1.              Crucially, could non-military teams of developers, with little or no personal combat experience and without the traditions and values inculcated into our military personnel, develop systems which are culturally sensitive to what the UK forces understand their roles to be? A particular danger may lie in developing applications which operate with an overly simple utilitarian calculus of costs and benefits, rather than seeking to retain as much of a humane ethic as is possible in a conflict.

 

  1.              This risk of buying into crudely utilitarian principles by default suggests that the design of AWS should be strongly influenced at all stages by our own service personnel and not simply out sourced to commercial developers. The whole process of conceptualisation, design and development should be approached as a close partnership between technical and military cultures and structures. Hence, so far as possible, in house procurement processes aiming for bespoke UK or NATO systems should be preferred to off the peg commercial solutions.

 

AWS potentially offer more ethical use than traditional weapons

 

  1.              International law has long sought to regulate and, so far as possible, to restrict the use of indiscriminate weapons. Land mines are an example which, once planted, are a risk to all in the vicinity and not just to intended targets. Reducing the indiscriminate capacity of weapons is therefore an ethically significant potential application of AI and an area where continued research and development should be actively encouraged.

 

  1.              This is an example where applications of AI could potentially be ethically advantageous, helping to address an area where some classes of weapons arguably cross an ethical line but are nonetheless routinely deployed by some regimes. Building a degree of AI-driven discrimination into the weapons, which enable them to be targeted with greater precision and intentionality, could render them morally more acceptable and greatly reduce the risk they pose to civilians, one’s own forces and unintended targets.

 

  1.         Successful deployment of such technologies could mitigate collateral damage, constraining the impact of war on people and societies at large. This advantage starts from the premise that indiscriminate weapons are ethically problematic and that political attempts to limit their use are only successful to the extent that international treaties etc are achievable and enforceable; a difficult proposition when they offer an aspect of military advantage. The potential of AI to reduce the indiscriminate effect of weapons is therefore a worthwhile objective. But of course constraints on the uses of AWS themselves will be subject to similar, and perhaps even more difficult, challenges of international agreement and enforcement.

 

Accelerating the OODA loop

 

  1.         Decision advantage arguably rests in the speed of a military commander’s ‘OODA’ loop. If they can Observe the situation, Orient themselves successfully to the threat, Decide on a course of action and then Act to enable it at a faster speed than their opponent, then they can hold the initiative and keep the enemy on the back foot. AI has clear advantages at accelerating this process from a military planning angle, able to observe and orient faster than human speeds, allowing military commanders to decide on the course to take and to act far sooner. A force equipped with an AI planning tool to augment human decision making will therefore find it far easier to gain and maintain the initiative. In this scenario, the AI may be used to present the humans with estimates and alternatives rather than to make decisions on their behalf. While this is likely to be an effective and important use case for AI in defence, it falls some way short of meeting the full definition of an AWS.

 

  1.         There are several scenarios, however, where the full OODA loop – including both the Decide and Act stages – cycles far faster than human speed of thought and action can possibly accommodate. The Phalanx CIWS (Close-In Weapon System) is used on ships to defend against incoming threats such as missiles, aircraft and small boats. In the case of a missile, the time from detection to actually striking the ship may be so short that a human could not hope to respond. The Phalanx must simply engage the target without human input or intervention. This AWS identifies the threat quickly and responds to safeguard lives. But it is currently a last resort solution, when there are no other options available. In an era of hypersonic missiles, this autonomous layer of protection arguably should be expanded out. An armed autonomous drone, for example, may be able to detect and engage an incoming hypersonic missile while a human could not hope to respond in time. Although still extremely challenging, an AI based system would be the best – perhaps only – chance to intercept successfully, giving a layer of protection that is otherwise unachievable.

 

Cyber and communications systems security

 

  1.         AI has huge potential use in protecting information systems and architectures. Cyber attacks on national infrastructure, including everything from military systems to energy networks to hospitals, increasingly threaten massive damage. A cyber attack, for example using bots to enact a DDoS (Distributed Denial of Service) attack, could be detected and countered much more efficiently and effectively by AI than by humans. As the attack could come from multiple infected devices across the world, an AI that could effectively and efficiently determine the nature and characteristics of attacks could autonomously safeguard vulnerable systems and allow for faster response.

 

  1.         Similar advantages would be applicable in electronic warfare, for example in terms of automatically jamming enemy communications far more efficiently and also protecting our own communications channels. Fixed Frequency Hopping is a standard defence against electronic jamming and sees radios moving across multiple frequencies every second. With human-led jamming, the only effective counter is to jam every frequency available, which is power-intensive and so limits the effectiveness of the disruption, or to jam specific frequencies being used which leads to incomplete jamming. An AI system could target specific frequencies far more effectively while identifying which frequencies would be best used by our own forces against enemy jamming attempts.

 

Information operations

 

  1.         The ability to conduct information operations will have a clear supporting role in 21st Century defence, both in terms of intelligence analysis and in gaining and maintaining control of the narrative in the event of conflict. In much the same way that the Delta Force attack on Osama Bin Laden’s compound in 2011 was live-tweeted throughout, few military operations will take place without a civilian camera or other sensor in close proximity. This provides a huge amount of data that could quickly become valuable intelligence. But for information to become usable it must first be seen by the right analyst and quickly enough for it not to be outdated. An AI that could monitor and actively search multiple channels, including social media, for relevant information and prioritise it for an intelligence analyst could vastly speed up the intelligence cycle and also further enhance the OODA loop with timely and pertinent information.

 

  1.         AI systems such as Large Language Models (cf. ChatGPT) could also generate social media content to help counter false narratives being promulgated by the enemy. In an era of ‘fake news’, the ability to respond quickly and accurately is key to ensure any malicious narrative against an operation or campaign does not gain momentum. An AI that could search through the huge pool of data that is contained on multiple social media platforms, identify potentially harmful ones, and then generate potential responses for a human user to implement would allow quicker control of the narrative.

 

Austere environments

 

  1.         While deploying a human into an environment will likely be preferable or necessary for the foreseeable future, austere environments could present a situation where an AWS would be a better choice. Already in environments such as firefighting, bomb disposal and nuclear reactor maintenance, a robot may be used initially to determine the situation while the threat is unknown or there is a likely significant risk to human life. A military operation in austere environments could evoke similar considerations where, for example, casualties would be unable to be evacuated within acceptable timelines.

 

  1.         Military history is littered with examples of rescue missions resulting in additional casualties, such as 1993’s Battle of Mogadishu. A rescue operation conducted by AI-enabled robots offers clear benefits for safeguarding lives, and a deployment of an AWS to an environment where the ability to evacuate as necessary was uncertain could see an immediate military presence on the ground until the necessary infrastructure is in place to deploy humans more safely. Given the reach of the UK around the world and the possibility of the arctic regions becoming a new global military focus, this would enable rapid initial military projection while mitigating risks to human life.

 

Conclusion

 

  1.         There are cases where the use of AI can be ‘good’ or ethical in defence. Indeed it is arguable that it could sometimes be immoral not to use it. As with many defence or dual use technologies, AI may be used in protecting life as much as in taking it, and in some scenarios there is therefore a moral argument to hand over decision making and acting to AI in situations where a human cannot be effective. Provided the design, development and deployment of AWS are underpinned by our values, they may be ethically acceptable and indeed potentially ethically imperative. To the extent that they may be ethically problematic, we are reliant – as with current WMDs – on brokering and enforcing international agreements. Hence, adopting a national position, in isolation from the geopolitics of both our allies and our potential adversaries, is not adequate or appropriate.

 

Professor Eamonn O’Neill

UKRI Centre for Doctoral Training in Accountable Responsible and Transparent Artificial Intelligence (ART-AI)

April 2023