final logo red (RGB)

 

AI in Weapons Systems Committee

Uncorrected oral evidence: Artificial intelligence in weapons systems

Thursday 21 September 2023

12 pm

 

Watch the meeting

Members present: Lord Lisvane (The Chair); Lord Browne of Ladyton; Lord Clement-Jones; Lord Bishop of Coventry; Baroness Doocey; Lord Fairfax of Cameron; Lord Grocott; Lord Mitchell; Lord Sarfraz; Lord Triesman.

Evidence Session No. 15              Heard in Public              Questions 201 - 207

 

Witness

I: Andrew Kinniburgh, Director-General, Make UK Defence.

 

USE OF THE TRANSCRIPT

  1. This is an uncorrected transcript of evidence taken in public and webcast on www.parliamentlive.tv.
  2. Any public use of, or reference to, the contents should make clear that neither Members nor witnesses have had the opportunity to correct the record. If in doubt as to the propriety of using the transcript, please contact the Clerk of the Committee.
  3. Members and witnesses are asked to send corrections to the Clerk of the Committee within 14 days of receipt.

9

 

Examination of witness

Andrew Kinniburgh.

Q201       The Chair: Good morning, Mr Kinniburgh. I am so sorry that we have delayed the start of your evidence. As you will have seen from the back, we have been trying to cover quite a broad canvas, and will continue to do so with your expert help. Could you begin by giving us a feel for Make UK Defence, how it works and what it aims to do?

Andrew Kinniburgh: We are very grateful to you, Chair, for inviting us to give evidence this afternoon. It is a great opportunity for Make UK Defence. We are part of Make UK, which is the voice of British manufacturing in the UK. It is a national business organisation that pulls together all the views of manufacturers across the UK and presents them to the Government. We do some lobbying work. We do a lot of policy work. We are trying to encourage the Government today to reintroduce on a national basis an industrial strategy for manufacturing.

Make UK represents around 23,000 companies in the manufacturing sector, which is about 1 million manufacturing jobs out of a total of 2.7 million. Make UK Defence is the small cousin of Make UK. All of Make UK is owned by our members. Our constitution is a subscription model. Our members pay a subscription to be a member of Make UK. By doing that, they become a part owner of the business as well. We are a not-for-profit business. Any profits that we make are reinvested in the business.

Make UK Defence is small but beautifully formed. We have around 400 members. We are growing quite quickly at about 20% a year. It is the same subscription model. Our particular specialism is focusing on the defence supply chain. We work very closely with the big businesses—the big primes—to help them improve and develop their supply chains and to find new suppliers, or perhaps alternative suppliers. We work very closely with, and without being too arrogant we hope that we are the voice of, the small and medium-sized businesses in defence. They are the ones that do not have a voice. That is where we come in, to represent them and, hopefully, strengthen their voice.

The Chair: Through that prism, what is your view of the current government policy towards the procurement of AI and AI-enabled or supported weapon systems?

Andrew Kinniburgh: I should point out at the beginning that I am not an expert in this. I describe myself as knowing very little about a great deal. Clearly, I have informed myself for this session.

The Chair: What we are after is the sort of feedback that you get from companies that are trying to engage with the Government in order to supply equipment, or in order to understand what the Government’s operational requirement may be.

Andrew Kinniburgh: Procurement generally in the UK is very skewed towards the big primes. We have just come back from a big defence show last week called DSEI at ExCel in London. It is the world’s biggest defence show. From talking to officials, parliamentarians and industry, it really feels that there is a gentle breeze of change coming through procurement. We are beginning to feel that this may be the time for the SMEs and mid-tiers in procurement.

Regarding AI specifically, the temperature that I have taken with our members and with other contacts is that defence procurement in the UK is hopelessly outdated for the world of AI. It moves, if you are very lucky, in months, and if you are, probably, regular, in years, by which point AI has moved on immeasurably and it simply is not fit for purpose, particularly when you bring it into the big primes, which can be quite slow and cumbersome; Defence Equipment and Support and the more traditional procurement routes.

There are some pockets of excellence in the Ministry of Defence. By the way, I am certainly not dissing DE&S and the other procurement arms of MoD; they do a great job. Organisations such as DASA, the Defence and Security Accelerator, are much nimbler and far more able to bring companies quickly into the defence supply chain and to work more dynamically, which is where AI needs that kind of support.

Q202       Lord Clement-Jones: We are all familiar with the Commons Defence Select Committee criticisms about the bureaucracy of the procurement process. In a sense, the underlying thought of the committee is that we need to bypass some of that bureaucracy in the age of AI. You talked about DASA. Have you come across the US Tradewinds Solutions Marketplace? We have been told by a US witness that that has been a way of the US DoD basically bypassing the bureaucracy in its own procurement system, which he did not minimise and sounds pretty equivalent to our level of bureaucracy, quite frankly. I wondered if you had come across that and whether Make UK has come up with its own proposals for a bespoke form of procurement for AI weapons.

Andrew Kinniburgh: No, I am afraid I have not come across that specific organisation, so I cannot really compare. You are absolutely right. We are probably quick in the UK to look in the mirror and beat ourselves up about bureaucracy and the way we do things and the way we buy stuff in defence, but many countries have similar issues to us, including France, Germany, the US and so on.

We have not come up with a construct that particularly focuses on AI. The current very strong bias—I do not mean bias in a negative way but just the way that the market is structured at the moment—is that the big companies tend to grab the big lumps of money and the little bits that fall from the table are where the SMEs kick in. In my research for this session, I spoke to a number of SMEs that are doing some really good stuff and some really innovative things, and that is where DASA and other organisations come in.

If I may expand that, the front-line commands have their own innovation units, and those, as far as I am concerned and my members are concerned, are largely impenetrable. They do not seem to have a front door, and that is a great concern for me. When you are a small business trying to traverse that very complex market anyway and the front-line commands are spending not huge amounts of money but significant bits of money in AI and in other leading-edge technologies, the SMEs cannot get in there and they cannot produce the good ideas and generate the thinking of an SME.

Lord Clement-Jones: Before the Chair hands over for our next question, can I probe you a little bit about front-line commands? At what level would you say that was? Is it at MoD level or further down the chain of command, at Northolt or whatever, in terms of operations? That sounds like a really interesting aspect of the nimbleness required and the tactical use of weapons.

Andrew Kinniburgh: They sit within the MoD main building, so they sit within head office, but they are certainly part of the front-line command. The honest answer is that I am not entirely sure where some of them sit. We have the rapid capabilities office for the RAF, which is doing some great stuff. It is working with some of our SME members—for instance, a company called Aeralis, which is a fast jet training business that is doing some really innovative work, but it is difficult to find the front door.

If we were able to harness DASA and its nimble fleet-of-foot approach and more SME-friendly approach together with the front-line commands and organisations like the Future Capability Group, which is an R&D science and technology group within DE&S, Defence Equipment and Support, and if we could find some way of linking all of those together and opening them up to the SME and mid-tier community, we would be much more match-fit in AI and how quickly that market is moving. That would be one suggestion that we might come up with.

Lord Clement-Jones: We clearly need to follow up on some of those acronyms. Thank you.

Andrew Kinniburgh: Forgive me.

Lord Clement-Jones: No, it is very useful. Thank you.

Q203       Lord Mitchell: I am slightly amused by what you are saying about SMEs not being able to find the front door. I would guess that if we were having a discussion on the NHS the same discussion could take place. It is funny that it plagues our Government, and in particular the Civil Service, that SMEs just cannot get in there. Anyhow, that is another discussion.

I am going to ask you a question. Based on what you have said, I think I can probably answer it word perfect, but I will ask it all the same. In your opinion, are the existing procurement processes capable of dealing with the rapid pace of development of AI systems?

Andrew Kinniburgh: No.

Lord Mitchell: Thank you.

The Chair: That is the most splendidly concise answer we have had for some considerable time.

Lord Mitchell: Very clear. Well done.

Andrew Kinniburgh: May I make one tiny point? This was a moment of clarity for me in building up to this evidence session. I spoke to a very small business that is working in the synthetic training environment, and it gave me an example that kind of made me think, “I think I get AI now”. Forgive me, I am sure there is huge expertise around the table.

The example someone gave me at that business was flying training. He said that they had an example of 39 pilots each flying a mission for one hour—39 hours of data, in effect. That produced 197 million data points. He said that AI can play a role in analysing those 197 million data points and can spot behaviours or the way the pilots react and potentially stream them into multi-engine, single-engine fast jet, reconnaissance or UAVs. For me, that was a really useful analogy to understand how AI works. Forgive me, you probably know all that already.

The Chair: That is a really vivid example. Thank you very much.

Q204       The Lord Bishop of Coventry: Thank you very much for your evidence thus far. Perhaps I could give a bit of context. A number of our witnesses commended the Ministry of Defence for its ethical principles, as outlined in its paper Ambitious, Safe, Responsible. A number also said that it was a good first step but it should not be the last. They have been calling for the application of those principles in operational practice. Clearly, procurement is an absolutely critical stage of good practice.

The MoD claims to provide teams developing technologies and solutions “with clear frameworks to support the early identification and resolution of safety, legal and ethical risks”. From what you have picked up from the industry and the relationship between industry and the MoD, do you get a sense that that is happening? If it is happening, are frameworks enough, or does it need another level of detail?

Andrew Kinniburgh: Thank you, Bishop. It is a really interesting question. I thought of you as I spoke to Roke, which is an AI and cyberspecialist business within Chemring plc. I was speaking to its chief engineer and he used the phrase “ethical but lethal”. I guess that is the kind of world we are dealing with, particularly with AI in weapons systems.

I think the MoD has a very strong legal, ethical and moral framework, actually. Another of my advisers told me about the ethical and moral team at Sandhurst who do nothing else but think about the moral and ethical side of warfare. We need to keep working on the philosophical side of things. There is a very tricky balancing act between lethality and the ethical side of things.

The frameworks are strong for the UK. You could argue they are too strong perhaps in terms of safety. Certainly, a couple of the people whom I spoke to in the lead-up to this session talked about the Americans—it is a bigger country, so it has larger ranges—sending uncrewed air vehicles, uncrewed surface vessels and uncrewed underwater vessels, and they have been working for years on that. They may not have all the answers and they may not have all the AI tools that they need today, but what they are doing, which is vital in AI, as you all know, is gathering data; they are gathering huge amounts of data from the real world.

The impression I got from the people I spoke to was that perhaps the UK was a little bit conservative regarding safety. The safety cases were very well put together and extremely robust, but actually there is a risk that we need to take to enable us to gather that data, so we need unmanned surface vessels bobbing around the Solent in a very busy part of the sea, and we need unmanned, underwater systems and UAVs. Our regulation is very strong, which in many ways is a good thing, but when it comes to leading-edge technology, particularly when AI is involved in decision-making, we get very reticent about granting permission, and that holds the UK back in gathering essential data even when the AI tools do not exist today.

The Lord Bishop of Coventry: That is really helpful, and I can very much see the point. I am just trying to get my mind around how the relationship works between the MoD and the bodies it is procuring weaponry from and working on and expecting to develop weaponry. Is it a dynamic, ongoing relationship? Does its team sit down and accompany a team from industry so that they are working together, or is it just, “These are the principles. This is the framework. You’ve got to fulfil it”?

Andrew Kinniburgh: It is a mixed picture, to be honest, Bishop. There are elements of that. There are organisations, arm’s-length bodies, within the MoD, such as the Submarine Delivery Agency and Team Complex Weapons where there is a single-source relationship and one provider. With submarines, it is BAE Systems at Barrow. With Team Complex Weapons, it is MBDA and some others. There is collaborative working together. DSTL, the Defence Science and Technical Laboratory, is clearly very involved in both of those areas. The Atomic Weapons Establishment is as well, with the nuclear deterrent in submarines. However, it is not spread throughout MoD procurement.

In programmes such as Ajax and Boxer, Boxer is a good example; it is still running well. Ajax has had its problems, as we know. That is where if you apply the old-fashioned procurement maxim of, “Here’s a requirement. Deliver that. You’ve given us a price. Now, off you go. Don’t bother me again. Come back with 190 eight-wheeled mechanised infantry vehicles when they are ready, bang on the deadline that you promised us”, to AI and to leading-edge technologies, you are dead in the water. We probably need to move even beyond the more collaborative approach of perhaps the Submarine Delivery Agency or Team Complex Weapons to a whole next level where you can hardly tell who is MoD, who is military, who works for an SME and who works for a prime; they are just working together.

Interestingly, I spoke to one of the world’s leading AI specialists in Ukraine, a gentleman called Anton Skrypnyk, who is chief executive of Roboneers, which has a very strong background in robotics and AI. I talked to him about his experience in Ukraine. Obviously, wartime drives different behaviours, and, as I would say, you take your cap badge off and leave it at the door. There is not that competitive element in procurement. He said that the Ukrainians have formed a committee of 50 AI experts from across the country and from across different sectors—healthcare, finance, defence, security and various other specialisms—and they work as a coalition, and the companies represented there also work as a coalition.

Basically, because it is wartime and because there is no time for procurement processes and going through the typical due diligence, they are working together as “Team Ukraine”. He said that the speed of development is frightening. They are watching their uncrewed air vehicles on the front line with software coders changing the code before the UAV has landed again back at base. It is a very dynamic situation. I know that is wartime, and, clearly, we have shareholders and competitors and others, but it was an interesting contrast to the UK.

The Lord Bishop of Coventry: That is very helpful. Thank you.

Lord Fairfax of Cameron: This is a very quick supplementary in relation to what you have just said. Do you think there are fully autonomous weapons systems operating in Ukraine at the moment?

Andrew Kinniburgh: That is a very good question. I honestly do not know the answer. I would be surprised if there were not, but I honestly do not know.

Lord Fairfax of Cameron: That is interesting.

Q205       Lord Sarfraz: Thank you for the nugget “ethical but lethal”. I really like it. We have found a political slogan for our election campaign next year, if nothing else. What we cannot do is expect the MoD to have the capability in-house and individuals who are up to speed with all things AI all of a sudden, right?

Andrew Kinniburgh: Of course.

Lord Sarfraz: It takes a lot of time to train them and bring them in. We have heard about the defence AI skills framework that is being developed right now. Are you seeing those skill sets improve? Do you have any recommendations on how they can fast-track some career pathways or any insights on building up the skill sets within the MoD for procurement of AI?

Andrew Kinniburgh: That is a very good question. If I may, I will broaden it to cover manufacturing as well. If you would indulge me for a moment, I will explain why. There is a programme running through MoD at the moment called the pan-defence skills framework, and the idea of that programme is to develop a common language—in engineering more generally, rather than specifically in AI—between the three services, which makes you kind of think, “Surely, we are working as one Ministry of Defence. Surely, there is a common language”. Apparently, there is not. When I asked whether industry was in there and the Civil Service as well, I was told, “Yes, on an as-and-when basis”. That to me beggars belief. We need a common skills language, and I imagine that AI is probably in a similar boat, although perhaps MoD, as it runs alongside the industry and AI experts, is using a common language. I think there is a lack of a common language.

I spoke to Vice Admiral Phil Hally, the Chief of Defence People, at DSEI, and he explained the more dynamic career management of the military and industry, and the much more dynamic ability to move in and out of the services, either as regular or as reserve service personnel, and in and out of industry. If we could develop a common skills language that we all agree and that industry agrees as well, it would make a huge difference.

Specific to AI, I am not close enough to it, to be honest, to give you a very knowledgeable answer. What I would love to see—this is getting slightly off the topic, forgive me—is flipping it on its head, putting an AI SME in charge and letting it try running maybe a smaller programme. Then it would bring in the specialists, rather than always defaulting back to the big primes and back to the big companies with all the same pressures and difficulties and bureaucracy of procuring from government. That is getting off the topic. Forgive me.

The Chair: Your remark about the size of the three services reminds me of the remark that was made at the time that the three service ministries were brought together in the Ministry of Defence. It was described as attempting to perform an appendectomy on a man carrying a grand piano, which may still have some relevance.

Q206       Lord Fairfax of Cameron: I think you have answered this question already, but it is here so I am going to ask it. Does the current procurement process in the UK allow for effective engagement in the availability of opportunities for smaller innovative software businesses?

Andrew Kinniburgh: No.

Lord Fairfax of Cameron: That is a no. There we are. Thank you.

Q207       Lord Browne of Ladyton: If I may say so, Mr Kinniburgh, you have a remarkable, verging on comprehensive, knowledge of the whole infrastructure map of the MoD in terms of procurement, which I suppose is essential for your job. I am going to test that a bit, for a purpose.

Andrew Kinniburgh: Not current, Lord Browne.

Lord Browne of Ladyton: Before I do so, the language issue bedevils the relationship between public policy and emerging technologies as well. We spent some time last week being briefed on something, and they used the word “platform” in ways that were verging on the incomprehensible. Almost everything in the room, no matter what shape it was, if it had technology in it, was referred to as a platform, but that is by the way. We could all do with trying to find common languages.

Attempts have been made to solve the problem of the SMEs. Government Ministers who work in this space, anything to do with the economy, talk about small and medium enterprises being the engine room of the economy. You know all this stuff, right? Attempts have been made in the Ministry of Defence, and I am thinking in particular of what I think is properly called the Defence Technology Exploitation Programme, which put together a modest sum of money, in terms of the sorts of money that the MoD can spend, to get the primes to work in partnership and collaboration with small and medium enterprises to build knowledge. Everybody tells us, when we talk about these things, that a lot of the innovation in this space comes from small and medium enterprises and small units. Have you any awareness of that, whether it is working or not, and whether it has developed at all, in fact?

Andrew Kinniburgh: I think it is so new, to be honest, Lord Browne, that perhaps we do not know yet. I have great hopes for it because I think it is the right thing to do and it is a bit more money being fed into SMEs. To give a shoutout to another trade association within the UK called ADS, aerospace, defence, security and space, it is, I think, managing that programme on behalf of the MoD. It won the competition to administer the programme. It has a lot of SMEs as well in its membership. I am cautiously optimistic, I think. The mood music last week was most definitely more SME-focused.

Lord Browne of Ladyton: Could I sum it up by saying that you are aware of it and you are optimistic about it, but as yet you have had no members coming to you and saying, “My experience of that was good”?

Andrew Kinniburgh: Correct.

Lord Browne of Ladyton: Okay. Can I draw your attention to something else? It is quite well known that DARPA in the United States was a great driver of innovation. It only had one customer, but it was a customer with a lot of money, which was the DoD. We copied it. We now have ARPA. It is in its development stages, I understand. It is beginning to take shape. Is your organisation, and you as the chief executive, in touch with these people in any sense?

Andrew Kinniburgh: We are in touch with most of these teams. We have not had any contact yet with the NATO organisation, DIANA, about looking at NATO-wide programmes and potentially bringing SMEs into that area.

We talk in UK defence about the “valley of death”. DARPA is very good because DARPA supports a business from the beginning, from the very early TRL—technology readiness levels; you may be familiar with the acronym. There is a scale of one to 10, one being the kernel of an idea in your head almost, and 10 being fully operational, fully supported, ready to go and completely capable. DASA, the Defence and Security Accelerator, and other organisations, such as DTEP, help companies, particularly SMEs, to move their product or their service up the TRL scale. They help you take a piece of equipment or a service from perhaps TRL 3 or 4 up to 5 or 6. DARPA would then continue funding and take you right through to what I would know from an industry perspective as profitable trading: you have a customer, you are actually selling the kit, you have an integrated logistics support plan, you have a maintenance, repair and overall plan, and you are ready to go.

The problem in the UK at the moment is there is that gap—the valley of death. You can get support from DASA, which is really good at it. It is not trying to nick your intellectual property; it helps you to develop your programme, but then the funding stops, and you are at the mercy of the markets and the big banks. We talk to the banks a lot. They assure us that none of them is anti-defence. They are all very supportive and, “What can we do to help?” and all the rest, but their behaviours suggest otherwise. They are very unsupportive of the defence industry and very reticent even to offer overdrafts.

Insurance companies will not insure businesses if they have more than 10% of their business in defence. The valley of death becomes incredibly difficult to bridge from maybe TRL 5 or 6 up to 8 or 9 when you have a Ministry of Defence or perhaps a prime contractor buying the kit from you. How do you bridge that gap? The free market of banking and finance and insurance and other supporting elements is simply not stepping up at the moment, and that is a huge problem that we are not seeing resolved. We can haul over the coals as many chief execs of banks as we like, but if it is a voluntary code it does not seem to change.

Lord Browne of Ladyton: Finally, do you think your manufacturers, your members, would be helped if this alphabet soup of initiatives was blended down a bit and was much more straightforward?

Andrew Kinniburgh: Absolutely. I cannot think of the word to emphasise how much I agree with that. If you look at the Department for Business and Trade’s funding finder, you get about 400 different funds. Defence is less complex, but it is still much too complex. Bringing all of that innovation into one single, cohesive view would be amazing.

The Chair: Mr Kinniburgh, thank you so much. I am sorry that we have been under the tyranny of the clock today—

Andrew Kinniburgh: Not at all.

The Chair: —but you have added really substantially to our understanding and to our inquiry, and for that we are all most grateful.

Andrew Kinniburgh: That is very kind. Thank you very much and thank you again for inviting us. We are very grateful to you.

The Chair: Thank you.