26
Joint Committee on the National Security Strategy
Oral evidence: AI and quantum computing in the national security context, HC 628
Monday 7 September 2026
4.20 pm
Members present: Lord Sedwill (in the Chair), Lord Arbuthnot of Edrom; Liam Byrne; Sarah Champion; Bill Esterson; Lord Godson; Baroness Kidron; Edward Morello; Lord Tunnicliffe; Lord Watts.
In the absence of Matt Western, Lord Sedwill was called to the Chair.
Evidence Session No. 1 Heard in Public Questions 1 - 16
Witnesses
Dr Michael Cuthbert, Director, National Quantum Computing Centre; George Balston, Strategy Adviser, Alan Turing Institute, and co-founder, AVERI; Professor Kenneth Payne, Professor of Strategy, King’s College London.
Dr Michael Cuthbert, George Balston and Professor Kenneth Payne.
Q1 The Chair: Good afternoon and welcome to the Joint Committee on the National Security Strategy. Our Chair, Matt Western, is unable to be with us today, so with the committee’s agreement I have taken the Chair for this session.
Today, we are holding a one-off session on AI and quantum computing in the national security context, and we have an expert panel with us today. We are likely to be disrupted by votes in the House of Lords, and that means there will be moments or short periods when we are not quorate, so I will have to suspend the session briefly for that until colleagues return, and then we will just resume where we left off. Thank you for bearing with us on that.
If we can start, may I ask the witnesses to just briefly introduce themselves? Professor Payne, why do we not start with you?
Professor Kenneth Payne: Thanks very much. I am Kenneth Payne; Ken is fine. I am professor of strategy at King’s College London. Most recently, I was adviser to the Defence Committee for its report on AI procurement, so I am somewhat familiar with this process. The other thing that it is probably germane to what I will say this afternoon is that I was a global commissioner on the REAIM commission—I will mangle the acronym: responsible and ethical AI in the military domain—which is an intergovernmental process looking at that topic for a couple of years. I do not normally wear a tie, so I might just loosen myself off a little bit.
The Chair: It is quite warm in here, so please take your jackets off, loosen ties, et cetera, if you wish. Some of us have done that already, so please do so as well. Dr Cuthbert?
Dr Michael Cuthbert: Good afternoon, everyone. I am director of the National Quantum Computing Centre at Harwell. We are a national laboratory focused on quantum computing, bridging from academic research into industry, looking at the technology, hosting commercial prototype platforms, and seeking to utilise quantum computing in new application areas.
The Chair: Welcome and thanks for joining us. Mr Balston?
George Balston: Good afternoon. I am co-founder of AVERI, a non-profit working on frontier AI safety. I previously served as the co-director for defence and national security at the Alan Turing Institute, where I still serve as an adviser. I was also the founding director of CETaS—the Centre for Emerging Technology and Security—at the Turing, although, today, I will be speaking in a personal capacity.
The Chair: Thank you all again for joining us. You are probably familiar with the process, but, just to let you know, individual members of the committee will lead off on different topics, but others might chip in as we go. We will not ask all of you to answer every question. We will usually start each topic by indicating which of the witnesses we want to lead off, but we might ask you to come in and, if you have something you feel you should add, please do so. This is very much an evidence-gathering session.
I am going to kick off. Again, thanks for joining us. The national security strategy, set out in 2025, pledged to build the national capability for AI and, indeed, other frontier technologies, increasing our national capacity, accelerating adoption and advancing the understanding of security risks. How far do you believe the UK has progressed against that agenda? That is perhaps something that all of you will have a view on, so again, why do we not start with you, Professor Payne?
Professor Kenneth Payne: Thanks very much. I was very excited when I read that. There has been some significant progress over the last couple of years—the sovereign AI fund, for example, funding companies and doing exciting things such as Cosine.
The trouble is that the frontier has been accelerating away. One of many alarming charts is the one produced by an organisation called METR. It is on that that you can see the exponential curve. It measures the performance of the frontier models to do coding and how long it takes them to do a complex coding task with a reasonable degree of reliability. At the moment, we are just going into the steep part of that curve and the frontier’s coding capabilities are doubling every 100 days, so that is the target that you are chasing if you are chasing the frontier.
I read something, which is almost certainly out of date now, from either McKinsey or the Tony Blair Institute. I am useless with numbers, but this was a nice way of remembering what is going on here. They said that the amount of committed capital expenditure in AI companies is 220 Manhattan Projects. That sticks in my mind and is quite compelling. That is the challenge.
If you want to be at the frontier, the frontier is accelerating away from you, and my slightly pessimistic take is that we have no chance whatsoever of getting to that frontier, with the implications that follow on from that for sovereignty for the UK.
The Chair: Just to briefly follow up on that—and the others might address this point as well—does that really matter? Is it about chasing after the frontier, or is it about adoption? Is that the more important question?
Professor Kenneth Payne: I hope so, given what I said about chasing after the frontier. Adoption is really important. What you do with these models matters. We are an ingenious country and will find ingenious ways to apply these models. It is, no doubt, an important part.
The other aspect is resilience. What do you do if you lose access to these models? How do you continue functioning as a society? How do you continue thriving as a society in all possible domains? Those things matter, but, ultimately, there are some very significant geopolitical implications that flow from not being at the frontier. The recent Mythos example, brief though it was, suggested a little taster of what might come.
Dr Michael Cuthbert: It is fair to say that quantum computing and quantum technologies more generally are still at a much more formative stage than AI. That said, across the national security and defence community, there has been very strong engagement with the national quantum technologies programme and, increasingly, moves towards early adoption and prototyping. There are real-world examples of navigation with GPS denial through the Royal Navy, working with GCHQ on early prototyping, and understanding the capability of today’s quantum computer versus the crypto-relevant machines that we may come on to talk about in the future.
Important is the switch from government as the funder to government as a user, working not just in the defence arena but more widely across different government departments, and, again, that early adoption. There is some driving of proof of concept projects with the defence primes, where that is as much an educational process as it is validating the technology, but getting people within those organisations, building skills and building familiarity, as well as building collaboration and a network across the expert academic user community.
The Chair: We will pick up a couple of those points in later questions. Thank you very much.
George Balston: I do have a concern about the UK’s ability to keep up with AI and how that relates to sovereignty. Frontier AI is fundamentally a general purpose capability, so this means across the national security and defence communities. It is going to be useful in a range of applications, including data collection and analysis, investigations, and even intelligence gathering. The question that raises itself for sovereignty is whether, at the highest level, the UK’s capabilities that rely on AI can be cut off, leaving us behind our adversaries.
There are three questions that we need to ask that relate to sovereignty for specific deployments. The first is around security. Intelligence and defence agencies like to and want to put classified data into these systems, and so they need to be able to do so. Second, to Ken’s point, is resilience. Is access to the model guaranteed or can it be pulled by another country without our permission or even without giving us any knowledge of that? Finally, to your point, Chair, is capability. How capable do these models need to be?
The point that I would like to make to start is that not all deployments need the highest level of capability, security and resilience, but certainly some do, so the UK must have some capacity at the frontier.
The Chair: Can I pick up that last point? Again, your colleagues might want to come in as well on this. Do we currently have sufficient underlying computing infrastructure—industrial, et cetera—to support the level of sovereign capability in AI and quantum that you have just described?
George Balston: My view is that we do not. We have some state capacity, including the Isambard-AI Research Resource. Generally, these are focused on academia and research, whereas we are seeing a critical dependency across the AI stack being the ability to serve inference and provide access to powerful models.
I do not think that the UK has a significant amount of inference capacity. Costs for electricity are much larger here than in the US. We have much less capacity in terms of data centres here. Running a 1 gigawatt load in the US for a year would cost around $800 million, whereas, here in the UK, it would cost $2.5 billion, and so we are already a little handicapped in terms of being able to keep up on inference capacity. I cannot emphasise enough that inference capacity is critical, and this is just for inference, notwithstanding capacity needed to train our own models.
The Chair: Perhaps just on the AI side, Professor Payne, and then, on the quantum side, Dr Cuthbert, do you have anything to add?
Professor Kenneth Payne: We come up short across the stack, as it is called. Our energy costs are high—high compared to France, for example—and the amount of compute that we have for training models and for inference is limited. We do not control any of the choke points in that stack because we do not make the high-end computer chips that are needed by these models, and we do not, unlike the Dutch, make the lithographic equipment that goes into making that chip, so we are scrabbling for some sort of hold on that stack. What could we offer if, as a country, our strategy here was to enmesh ourselves in a broader ecosystem of frontier models? It is not clear that we have a particular locus at the moment.
Dr Michael Cuthbert: Again, given that quantum technologies are at a much earlier stage, we perhaps have a stronger position within the supply chain and within the full stack. Indeed, global investment into quantum technologies is, of course, dominated by the US and China, but the UK plays very strongly as a third player, having had a very mature national programme over the last 12 years.
A weakness within the supply chain is device fabrication. Where will the chips come from, whether they are prototyping chips at this stage or large-scale manufacture in due course? Most of our clean rooms have an academic research focus. We have particular strength in the photonics industry, particularly across the central belt in Scotland, and that would be a key enabler for all quantum modalities in the longer term. We have real strengths in specialist manufacturing, particularly around Formula 1, which is often a route for small-scale fabrication into the quantum ecosystem and the cryogenic supply chain.
Within the software stack, we have particular strengths in quantum software error correction, which is a crucial part of the requirement for quantum computing, applications and algorithm, as well as a very strong research base. We have a much stronger grip on the quantum stack, but that is always fragile and something that we have to maintain the nurturing of.
Q2 Liam Byrne: We face these dilemmas in the way that many other middle powers do. One of the assets that we still have, just about, is the ability to generate alliances. If I think about our partners in the EU, our GCAP partners such as Japan, our AUKUS partners such as Australia, or the partnership that is now deepening with South Korea, can you begin to see a way in which almost a middle-power stack could be assembled through deeper and better co-ordinated work with some of our allies? Professor Payne, maybe you could kick us off.
Professor Kenneth Payne: Yes, potentially so. Even if you add up all the venture capital and all the investment funds available to those middle powers, and roll in Singapore and other countries that might be similarly like-minded to act together as a counterweight to this bipolar world, you come up short of the amount of venture capital that America has thrown at the problem or has been able to generate to fund these companies.
I would caveat my answer to Lord Sedwill as well by saying that that is the architecture as it looks now. It may not be how AI is done in the future. There is research going into world models that may have different computational requirements. There is research going into different chip sets that may do things rather differently from the way that modern GPUs do. With all these debates, we need to be careful not to be too anchored in where we are now rather than lifting our eyes a little bit to the future. That is perhaps slight reason for optimism.
To your point about an alliance of people, I and many others have been making this point for a number of years now. There has been no progress—
Liam Byrne: No progress?
Professor Kenneth Payne: That is perhaps unfair, but no progress along the lines of developing a middle-power frontier capability. Let me put it that way. The countries that have tried to do so have largely done it unilaterally. Germany had a brief punt at it. France had a brief punt at it with Mistral. It is now falling behind the frontier with its latest models. I do not see a collective decision to develop frontier capability being made just yet.
Liam Byrne: Is it fanciful to think that such a thing is possible?
Professor Kenneth Payne: No. If we revisit the question in several years’ time, the consequences of our dependence on American technology will have become much starker than they are today, given where the capability of the models is.
Liam Byrne: When I have this conversation with our allies in countries such as Japan and South Korea, they often implore the UK to be more assertive in coming forward and providing some leadership for this kind of initiative. Do you think that the UK is providing the right level of leadership for this sort of initiative today?
Professor Kenneth Payne: The level of ambition that has been displayed in the last couple of years has not been adequate for the requirement that we have as a country.
Liam Byrne: So it is not a fanciful idea. It is possible. Not a lot of progress has been made, and UK leadership is not in the right place.
Professor Kenneth Payne: That is correct.
Q3 Lord Watts: You talk about the limitations of the UK in comparison to the United States. What about the skills base? As I understand it, in Silicon Valley, engineers are being paid £3 million to £5 million a year, which is a lot of money for anyone to provide. Secondly, you talk about America, but what about China? China seems to be trying to open up its systems to the world, whereas America is restricting them. What are the lessons to be learned from that?
George Balston: I would say that, on expertise and skills, the UK definitely has some strengths. We have some incredible capacity to develop AI specialists in both technological and governance careers. One of the challenges to that is that, once we develop the expertise, that is often moved to American-owned or foreign-owned companies. It is great to have that expertise in the UK. We currently have much of Google DeepMind’s staff here, although my understanding is that that is reducing. We have a very strong AI Security Institute, which does some fantastic work on evaluations. There is a synergy from being able to have all those people in the same place, but we are missing out at the end of that academic pipeline, where talent will often go to other countries.
In terms of the US and the Chinese approach, you are correct that, generally, the frontier models that the US is developing are what we call closed weight. The companies that develop the systems and the models provide access to those themselves, and the models are proprietary, whereas the Chinese developers are producing lots of incredibly powerful open-weight models. That means that anyone in the world can go and download them from the internet, and, as long as they have sufficient hardware, use those models. They are still spending astronomical sums on developing those, and they have a very different set-up for doing that.
To the previous point, I saw a fantastic article on what would need to happen for a middle-power frontier AI model. The costs that were attached to that over four years were half a trillion US dollars, $300 billion of which would be spent on compute alone. This article was by a chap called Anton Leicht. It talks through how much of that you might need to spend on compute, talent, et cetera. Those are the sorts of figures that we are talking about, so not impossible but fairly large.
The Chair: We are going to come back to the skills question more generally in a moment, so we can just park any further comment on that for now. Dr Cuthbert, we will come back to that on the quantum side as well in a moment.
Q4 Lord Godson: Just picking up very quickly on the encouragement to work with other middle powers, and since there is a lot of initiative to work with France, what is the reason why Mistral fell short? I would be grateful for the latest insights on that.
Professor Kenneth Payne: They simply have not raised enough money to afford the hardware that they need to train the models. It is as simple as that. They did not have as much money to spend.
If I may, I just want to add a very short follow-on to George’s comment about China. It is perhaps a little unfair, but I am always reminded when thinking about open-weight models that your first hit from a drug pusher is always free. Downstream of that, you may find that there are consequences. Perhaps we can come back to that.
The Chair: That is a vivid comparison. Thank you. We are going to come on to dependence on foreign infrastructure and cloud providers as part of this in a moment.
Q5 Lord Arbuthnot of Edrom: If I could just come in very briefly on sovereignty, there is a book called Ghost Fleet by PW Singer and August Cole, which is based on the principle that many of the chips in our critical defence infrastructure that we rely on are built in countries that do not have our best interests at heart. Is this a serious problem?
George Balston: I would say so. To Professor Payne’s point, the global supply chain for AI hardware is geographically diverse. This is a strength. There are lots of choke points across the stack. You have Dutch lithography. You have chips being assembled and packaged in Taiwan. You have some UK intellectual property in the chips. You have American chip design. You have gases being developed that are used in the manufacturing process in the UK. There is this very geographically diverse stack.
Theoretically, that means that the risks of those being cut off are reduced a little because there are quite a lot of dependencies in there. I would say that few of the important dependencies are British at the moment.
Lord Arbuthnot of Edrom: I was talking about something else. I was talking about having our F-35 aeroplanes switched off mid-flight.
Professor Kenneth Payne: I cannot speak to the F-35, but I would be somewhat reluctant to have Chinese cars in secure locations, for example. As a country, we perhaps need to think about how reliant we are on these technologies across a range of medical transportation systems as well.
The Chair: As a point of information, in many secure defence locations, they are not permitted inside the wire, in case they operate in exactly that way. That goes to some brands that we would not naturally think of as Chinese but have a lot of Chinese kit in. On this point of foreign infrastructure, Baroness Kidron, you can take us further into that.
Q6 Baroness Kidron: Question 1 took us right across, and we are going to come back now. I would like a little more detail on the question of dependence on foreign Governments. We have heard about loss of access and the security risks, but it would be very helpful if you were to talk about the things that most worry you about the dependence. Maybe we could start with Ken, then George, and then Michael on the quantum side.
Professor Kenneth Payne: For me, it is not so much where we are at this moment, but where we are going to be in two years’ time.
The Chair: Apologies; order, order. I am afraid we will have to break for a few minutes for a vote. Since you had only just started answering the question, we will just pause it there. We will then come back to the question in a moment.
Sitting suspended.
Baroness Kidron: What I was really trying to get at is a bit more detail about what you think the problem of dependency on foreign states is, first of all in AI and then in quantum.
Professor Kenneth Payne: My concern is less China and more our relationship with America, which produces the best models at the moment. Later we might get into how China conducts some of its research in these models, but, for the moment, it is behind where America is in making these models, and I see that continuing broadly.
There are two areas where, from a national security perspective, I am concerned about that reliance on America. One is cyber capabilities. In recent days, we have become aware of just how potent a risk that is going to be with the Hugging Face hack by an OpenAI model, autonomously, demonstrating hitherto unknown abilities to escape its sandbox and go off and do its own thing. There is more of that coming our way.
The other finding from recent months has been, again, from OpenAI, which is solving hitherto unsolved maths problems, persuading leading mathematicians that it is demonstrating creativity beyond what we have seen so far, and producing novel findings.
Both of those are of great use in the cyber domain as offensive tools, and, with those two episodes, we are just starting to get an intimation of what might be coming for us. That is a threat across the whole spectrum of societal activity, not just in defence.
The other one is the use of language models as part of situational awareness and targeting systems. You may have heard of the Maven Smart System, currently being adopted by NATO and developed in America. There is a very good book recently on that, called Project Maven, which I commend to you.
Language models have been, for the last several years, embedded in Project Maven. It is not clear from this side of the security line what it is that they are doing for the targeting, but they are doing something—providing context, I suspect, developing courses of action, and so forth. If you are reliant on models to do that, and if they confer what the military calls fighting power—and I think that they do—it becomes desperately important to have them, and we are exposed to our ally in that respect.
Baroness Kidron: On your first point, you said that more of that is coming our way, and I absolutely agree with you, but I am interested to have your thoughts on what we should do about that. What are the moves that you would make in this circumstance, particularly around OpenAI, but more generally?
Professor Kenneth Payne: There are two things, since I have said that we are not going to create our own. One is to enmesh ourselves as best we can in that process. It may be expertise or it may be another part of the stack, but we had better find one, or more than one, and develop it. The other is to develop resilience. We need to find a way of protecting ourselves against these sorts of autonomous cyber agents. Both those things are extremely challenging to think of.
You will perhaps have seen the reports getting some coverage today from OpenAI’s chief scientist. It is called something like “an alien intelligence”. Open AI has an IPO coming, and an interest in talking up its product, but I do not think he does personally. The quotes are fairly sobering from him: “We will actually see machines meaningfully smarter than ourselves in our lifetime”, and that is conservative. “No one is prepared”, he says. “This is a time that calls for extreme caution”. That is where we are. I wish I had the answer and could say, “We need to do this, that and the other”, but our options are somewhat limited. The threat is in the cyber domain, principally.
George Balston: I would agree with Professor Payne that my nearest-term concern is cybersecurity. We are seeing the ability for the frontier models to have incredible capability in cyber, in an offensive and a defensive way. We have seen some of the offensive way through some of the loss of control incidents that have happened over the last few months. We need to reorient the conversation around how we can use these defensively as well.
The challenge with using a frontier American model defensively is that, off the shelf, it will refuse to do much cyber activity because it would breach the safeguards that exist on that. You would want to have direct relationships with the company so that you could have models that had some of their safeguards removed, so that they could be optimally useful.
The mitigation to that might be using Chinese models to defend our critical national infrastructure, which there will be an understandable squeamishness about, but it might be a more resilient option than relying on models that could be pulled from servers and we would not be able to have access to.
My emphasis on cyber is that we need to have a diverse approach that includes working with strong partners, as well as some things that might seem quite unpalatable to us at the moment, using Chinese models to help us in that.
The other thing that I would say I am significantly concerned about in the more medium to longer term—when I say “medium to longer term” in AI, I mean months, not years—is recursive self-improvement. You have to contextualise the messages that frontier AI companies put out with the situations that they are in, but they are beginning to talk about having automated their research. Large parts of the research that leads to improvements in AI capability are now undertaken by AI agents, and humans are monitoring that. That, to me, seems like an incredibly concerning situation and one that will need strong governance efforts to counter.
Baroness Kidron: We will be coming back to the governance efforts.
Dr Michael Cuthbert: In quantum computing, the development cycle is in a different place. In terms of overseas dependencies and interdependencies, the No. 1 concern that I would have is investment. Patient capital in the UK, and in Europe more generally, will simply never be able to keep up with the level of capitalisation available in US markets.
Acting as a significant driver there are some of these international programmes, such as EuroHPC, the UK ProQure programme, which we may say a bit more about in a moment or two, and the US DARPA QBI programme. All of these are set to verify and validate performance of early-stage prototype quantum computers.
The DARPA QBI programme has an investment ticket of $300 million per company if you qualify through to stage C, and that is proving to be a very significant lever for companies both to get the validation for their end users, which drives commercial success, and to anchor themselves into the US, driven by US Government investment. The US CHIPS Act has also made $2 billion available very recently, partly to solve technical issues, but partly to drive acquisition. That certainly starts to put some of the European innovative start-ups at risk, so investment is a significant issue.
From a more technical perspective, some of the critical materials and minerals in device fabrication are risks for the wider European quantum technologies, not just quantum computing developments. Materials such as helium-3, caesium, strontium and rubidium are all absolutely at the core of these quantum technologies. While there is an open market for these today, even if there are bottlenecks on supply, that does not mean that it stays like that for evermore.
Baroness Kidron: What I am hearing from all of you is that we are way behind, and that there is not enough money in Europe to invest to catch up. That is the top-line message, and obviously the detail is important, but I am interested to hear from you whether you think that Europe as a market has some value to America or to China that could be a negotiating point. At the moment, we have not explored what that access to market means and whether that gives us something. Also, maybe thinking more creatively, where are the potential choke points that you can see—however nascent, in a sense? Otherwise, we could just pack up and go home. Maybe you could each just say a little about that thought.
George Balston: On the first point, on the EU as a consumer base—
Baroness Kidron: And beyond—middle-income—
George Balston: —that is strong leverage. Fundamentally, these closed-weight frontier AI companies need to make money, and the EU and outside of the US is a major market for that. The EU has been fairly forward-leaning on governance efforts, and we will probably come on to talk a bit about this. It has a strong piece of legislation—the EU AI Act—which compels developers to do a set of things that contribute quite strongly to safety and security of AI systems.
So far, we have seen US companies seem quite happy with abiding by those. There was a code of practice that was developed with respect to the AI Act, and it was signed by the vast majority of major providers in the US. There is an appetite to have products and services on sale in the EU and beyond, including in the UK, and so banding it together, to some extent, does provide leverage there.
Dr Michael Cuthbert: For sure, significant sectors in Europe such as pharmaceuticals, advanced materials, financial services and telecoms provide a pool to companies from outside Europe. I would add a what-if question—what if quantum computers do not scale, or the next 10 years do not materialise as the industry road maps project? Europe already has significant partnerships and real strength around algorithm development. Therefore, we need to challenge ourselves on what we can do with a modest-scale quantum computer through algorithm enhancement, rather than assuming its brute force to just make a larger and larger-scale machine.
Professor Kenneth Payne: I would like to hope so. George is right that the EU AI Act is a very comprehensive and very EU-like, if I may say, piece of legislation, but I have not yet seen anything that I could point to concretely and say that the EU’s position has produced a change in behaviour in frontier labs in the States.
There was a very brief period where OpenAI introduced a new feature called advanced voice, which you are probably familiar with. You can talk to the AI using your voice. There was a view that this violated the provisions of the EU AI Act in so far as it prohibits affective computing—that is, computers that can recognise tone of voice and so on in the user, and not just the words—and so OpenAI did not roll out that feature in the EU for a few weeks. Perhaps that is an example, but it detained them for only a few weeks before somebody decided that they were going to press on with it anyway.
The EU is an important economic actor, and these companies need to make money, but they are still raising money like billy-oh from venture capital at the moment, so it is not clear that they are desperate to toe the line of what the EU says at this point, and I am not confident that we will arrive at that point any time soon.
Q7 Liam Byrne: The hyperscalers are running into quite a significant set of problems in the United States with getting data centres built, and the EU is a €10 trillion consumer market. How important is sustained access to the American hyperscalers, and is that likely to have an impact on their appetite for complying with the future of standards in the UK?
George Balston: I could talk a bit to the capacity for inference, as I mentioned earlier. Much of the capacity that we have for inference in the UK, by which I mean the ability to use frontier AI models, whether they are closed source or open source, is dependent on US hyperscalers, albeit in region. They have data centres in Europe, in the UK and elsewhere, but they are still owned by US companies and so are, jurisdictionally, reachable by and accountable to the US.
Going back to the situation that we had earlier this year with Fable and Mythos, for example, if a company was compelled to stop serving a model for specific reasons by the US Government, it would not help us having a non-sovereign hyperscaler serving that to us.
I have another concern about the Chinese market as well, which is that there has been discussion in the US on adding Chinese open-weight providers to the entity list, which would mean that US companies would stop serving their models, including US companies operating in Europe and the UK. Even EU sovereign inference providers would have strong pressure on them to stop hosting Chinese models if they had strong exposure to the US as well.
Professor Kenneth Payne: My sense from personal interaction with Americans who are, as the newspapers say, familiar with the decision on the Mythos withdrawal was sobering. When a colleague of mine put to them that this put the EU at disadvantage relative to American companies, the answer was extremely blunt and pugnacious, and I did get a sense of naked power in that exchange. It is an exchange; read into it what you will. Maybe the EU will come out ahead in that test of will in shaping American frontier companies, but it did not seem that way to me from that conversation.
Liam Byrne: That is interesting.
Q8 Lord Watts: If I could just very briefly touch on the Chinese thing, the American model is a closed one. The Chinese is an open one. What are the implications of China spreading its wings around the world? If it can offer something for free compared to the American constraints, is it likely that, over time, China would enjoy, not just in the UK, but across the world, dominance in there? I know that it is behind America at the moment, but it is catching up very quickly. It is a tactic of the Chinese to get it out there and make it free.
It reminds me of videotapes. The best recorders were Betamax. What happened is that VHR spread its wings, and everyone bought into it. It is not necessarily a question of whether theirs is better than the Americans’. If they flood the market, I would have thought that it puts them in a very strong position.
George Balston: Yes, absolutely. There is a really good point there. If a model is good enough for a certain task, the market or individuals who need to perform that task will use the cheapest models that they can to do that. A really important point here is that Chinese open-weight models are much cheaper than the US frontier.
They are also less capable; the estimates are put between four and six months behind, but to forecast that forward, taking the model releases that we are seeing today from American companies, such as GPT-6 Astra and Fable 5.1, we will likely see open-weight models with the same capability before the end of the year.
It is a very intentional thing that is happening, where it is an undercutting in cost, but a slight cut in capability. To my point that we should be less squeamish around using Chinese models, for many cases, if the capability is good enough, and you are able to have sovereignty of that capability because you have custody of the model itself, that might put you in a stronger position than being reliant on a stronger model that you cannot guarantee access to.
Professor Kenneth Payne: For many use cases, that would be fine. You will own the rack of machines on which the model sits, and you can take some reassurance from that, but there are some use cases where marginal advantage in the model performance is going to be critical, and offensive versus defensive cyber is one such domain. There, you would want to be at the bleeding edge. If your access to the bleeding edge depends on Chinese good will, you might find yourself in something of an exposed position.
As I say, for many use cases, the Chinese model is really good. Another thing that you can do with these open-weight models is not just to take them as they are, but to retrain what the weights do. Thomson Reuters, the news organisation, has retrained a large Chinese model for its own purposes, and I am sure that that is absolutely fine as a use case.
Q9 Baroness Kidron: This is a very interesting part of the conversation. Some tasks need the frontier, but so many tasks do not. That is not something publicly understood, or even understood around policy circles. Would it make us more robust and less dependent if we had a flourishing of smaller examples, such as the one from Thomson Reuters, which is fantastic? We are talking about resilience across society and not simply trying to keep the hackers out.
George Balston: There was an exemplar point on this with the OpenAI and Hugging Face incident. A very powerful OpenAI model tried and succeeded in hacking into Hugging Face. Hugging Face had to use Chinese models to defend itself; it could not use the American models, because it hit their safety filters, so you had a very clear case there where using a Chinese model was better than using none at all.
The Chair: That is an interesting point. Thank you. Let us come on to talent and skills.
Q10 Lord Arbuthnot of Edrom: I declare my interest as chair of the Thales UK advisory board. To a certain extent, we have covered the issue of talent and skills. If you each of you was asked to make three headline points as to how we can best ensure that we have the talent and skills for the development of AI and quantum, first, in this country and, secondly, in academia, what would your three headline points be? Dr Cuthbert, you mentioned DARPA and its incentives of $300 million per company. Do we have the incentives there?
Dr Michael Cuthbert: It is not directly related to talent and skills, but the UK equivalent programme in anchoring UK companies here and driving inward investment is called ProQure. This is a three-stage government procurement exercise—the first stage of R&D, the second stage of scaling, and the third stage of large-scale infrastructure procurement into the 2030s. That will drive employment and skills attraction into the UK, as well as attracting overseas companies, so that is certainly very favourable.
The challenge is around where these people are coming from. The pipeline through the traditional academic PhD route, which is typically where people gain their experience, is on a three-to-four-year cycle. We already have doctoral training centres, and it is tens of PhDs per year, not the hundreds per year that the industry needs.
My estimate is that there are about 2,500 people across quantum technologies employed in the UK today, both in academia and in industry. From our estimates at the NQCC, there are currently 140 jobs advertised today, and I expect to see about 200 people employed over the next 12 months and 500 over the next 24 months. Those do not sound like large numbers, but, if you project that forward 10 years, the entire industry needs to grow by a factor of 10 as a minimum in its level of employment, so that pipeline of skills is critical.
It is a mistake to see them coming only from the traditional route of quantum physics rather than other scientific disciplines, engineering in particular. Industry will scale on the back of engineering, as well as computing science, where, across telecoms and AI, there is also significant pressure for the same talent pool.
How do we retrain and reskill people from other disciplines, such as systems engineering and perhaps some of the electronic disciplines, who already have perhaps 10, 15 or 20 years of industry expertise, to give them a level of familiarity with quantum technologies that means that they can really contribute, rather than, as I say, view it as being only people with a PhD in quantum physics who can contribute? That is simply not the case. We have to knock down some of that mythology around the word “quantum” and just say that this is a new paradigm in computing. It is really exciting how we drive those STEM skills into the discipline.
Professor Kenneth Payne: I am not sure that I have anything desperately profound here, and I certainly would not add to your germane point about salaries. I do not think that we are going to be able to do much about the salaries that are on offer in the Bay area. They are nuts.
I wrote down three things. First, we need to find a way of strengthening the connection between the frontier labs and UK universities. The direction of travel—perhaps salaries are a part of that—has been to hoover talent from the faculty into the labs. We need to find a way of keeping that generation of research scientists who are at the frontier engaged with the next generation of students coming through. I am not 100% sure how you do that. STEM is wildly popular with 18 year-olds and the people going on to do their postgraduate education. The university sector in the country is not in robust health, however, and that is one part of the problem.
Secondly on skills and talent, one area where we are doing a good job of educating computer scientists is in educating the next generation of Chinese computer scientists. We have a lot of Chinese international students in the country, which is something that we might want to think about more strategically. The two interests of higher education and national security are pulling in different directions.
The final thing on skills and talent—and I was struck by this in my work with the Defence Committee—is that public institutions, whose job is to shape this discussion and scrutinise it, are in need of technical assistance. There is a skills gap to provide advice to bodies such as yours, to Parliament more broadly, and to organisations of AI governance that may not yet exist but come into existence. We have to find a way of providing appropriate technical support for these sorts of discussions.
George Balston: You asked for three points, and I think I have three. My first is to consider extending the AISI—AI Security Institute—model. In AISI, we have seen the creation of a very successful organisation that is globally relevant at the frontier. It has world-leading expertise on the ability to evaluate AI models. It is worth having a look at how it managed to do that—and it was successful in doing that—and then possibly trying to copy that model for the national security AI community as well, which we need much more talent and resources in.
The second is to emphasise things that money cannot buy, and there are some, depending on the motivations of the individual. There are two important things there. One is access. People like working for AISI because it gives them the ability to work at the frontier, working on models before they have even been released, which very few people in the world do—a handful at AI safety organisations based in the Bay area and a few at AI security institutes across the world. The second one is mission. Lots of people are very mission-aligned and would sacrifice larger salaries to be able to contribute to the mission of protecting and safeguarding the UK.
The final one is about talent. We need enough talent, but enough talent for what? My own view is that we do not need to be hiring the sorts of researchers who would have seven or eight-figure compensations to help us train AI models, because that is not something that is tractable for us to do. Rather, we need people who can take a model and then implement and integrate it into systems, and people who we can put through vetting and who can do this within the national security context.
Finally, and importantly on that point, we also need a much more informed leadership and middle management across our institutions, and particularly the Civil Service, so that the clarity and the mission around using AI is emphasised such that people think, “I want to work in AI. Either I could go and work for Google DeepMind or I could go and work for GCHQ”. Emphasising that is very important.
Lord Arbuthnot of Edrom: None of you has mentioned the point that 22% of computer scientists and workers are women. If it were an equal point between women and men, there would be a huge talent increase. Is there anything that we can do about that?
George Balston: I am very aware that I am sitting on an all-male panel in a mostly male room. There is a huge amount more that we can do. We need to make spaces for computer scientists and AI engineers feel much more welcoming across male and female. There is much to do there.
The Chair: Thank you very much. We want to move on now to global governance and how we are managing this collectively. Lord Tunnicliffe is going to lead us into that.
Q11 Lord Tunnicliffe: I have three questions that you might answer in general. Are the existing international institutions capable of managing AI-related security risks? The UN Global Dialogue on AI Governance met, apparently, in July 2026, and it is not going to meet until 2027. Are such institutions capable of achieving the right pace? Should there be international agreements governing certain applications of quantum technologies?
If you look at the last thousand years of international agreements, they do not have much of a record, frankly. They usually break down pretty quickly. They seem to work only when nations accept that trespassing outside an agreement is going to be equally painful, so that you are legitimising or codifying the status quo. As far as I can see—and I do not know much about this subject—the status quo seems to be all over the place at the moment because of rapid development. Is there any chance of genuine international agreements that people will obey, even where, for at least part of the time, one nation feels at a disadvantage?
Professor Kenneth Payne: As I said in my opening remarks, I was a global commissioner of this intergovernmental process on AI in the military domain. What is interesting is that that conversation, which started internationally back at the UN in the early part of the last decade, was rather separate from the Bletchley process on frontier stuff. Now they have in essence merged into the two or, rather, the Bletchley process has subsumed a large amount of the military discussions.
With both those discussions, I felt then and do now that there was very limited prospect of a binding international agreement, for a variety of reasons but principally the security dilemma. The two countries that could make a meaningful agreement have no interest in doing so. Most of the noise about the agreement comes from countries that have no real prospect of achieving cutting-edge AI and feel themselves to be increasingly exposed to the consequences of not having cutting-edge AI, especially offensive cyber benefits and broader fighting power benefits from it.
Legacy militaries suddenly look a bit clunky, which we saw in the American action with respect to Iran recently. Saying nothing about the strategic wisdom or otherwise of that campaign, you saw a demonstration at the start of what AI-facilitated military activity can achieve. For that reason, countries that cannot do it are worried and countries that can do it have no appetite to stop doing it. I do not see any reason why that is going to change.
The statements that came out of REAIM, the Convention on Certain Conventional Weapons or the UN General Assembly more broadly as part of this dialogue are exceptionally banal, which is the only way you can get all parties to sign up to them. So I am a sceptic.
George Balston: I would make a distinction here between international agreements and national progress on governance. International agreements, to start with, are not particularly well developed at the moment.
To Professor Payne’s point, sufficient incentives do not exist. There is a race going on at the moment between the US and China in terms of capability. There are some concerns about safety, which both countries share, particularly those around loss of control, given what we have seen over the last few months, but those are not yet strong enough to bring both parties to the table and agree an international governance regime.
That would also require technology that does not exist yet, such as inference-only data centres. Those are data centres that cannot train models but can only serve them. We have some expertise in this country building such hardware governance. Particularly, there is a company called Amodo Design, which does some great work in this area.
On national governance, there is good progress taking place, predominantly in the US, where there is movement at the state and federal level. At the state level, a few Bills are being passed and have already been passed that mandate certain safety requirements for developers. At the federal level, there is a Bill that has been introduced to Congress that would mandate things such as independent and regular compliance audits.
On the other side of the Atlantic, we have the EU AI Act, which, as we have mentioned, is a good governance regime. One of the things I would note to Professor Payne’s point is that Anthropic introduced watermarking on to its systems. My understanding is that that was directly relevant to the EU AI Act. That is an example of a US company making some changes to its technology for compliance with a European standard.
The final thing that I will say on this is that the UK does have a position of some strength in this with respect to its evaluation capacity. Translating that into standards and better governance for AI seems to me like an opportunity for the UK to play a big part.
The Chair: Dr Cuthbert, I know quantum is slightly different, but perhaps you could touch on the same issues and on quantum cryptography, which is clearly at the front of everyone’s minds.
Dr Michael Cuthbert: First, in terms of international co-operation, the area of standards is quite a rich area at the moment. There is certainly a lot of dialogue, particularly around the G7 countries with some other international members.
There is a specific aspect of quantum computing about the verification of the hardware as it develops, and, if I can refer to something beyond what is possible, through emulation, so the scale of computation that we can run on a classical computer and demonstrate that we get the correct answer, in simple terms. If we are thinking about simulating chemical species or a complex system, is there a physical environment that we can compare it to so that, when we have run a complex quantum computation that is not classically simulable, we can measure it by looking at the real world and seeing how nature has come up with the same answer? That is a vital part of international collaboration.
Beyond academia and the national metrology institutes, NATO has a real potential to contribute. In the area of quantum technologies, I worry that NATO is seeking to replicate activities of other national institutes, creating testbed platforms, software labs and application centres, rather than facilitating the activities of member states and helping to support and drive collaboration.
Coming on to cryptography, which of course is a key aspect of the threat landscape from quantum computing that sits alongside the opportunity landscape in terms of all the applications across many industrial sectors, there is a concern more widely. If somebody had a crypto-relevant machine, would any of us know?
That is really where very tight international collaboration between like‑minded countries is absolutely vital. As algorithm breakthroughs are made or as the technology scales to large-scale crypto-relevant machines, which would still appear to be a decade away, we need to have those ever-tighter international collaborations where we are able to share our understanding of how that crypto-threat is materialising or not.
Q12 Baroness Kidron: I have a small question. You mentioned the States coming in with a flurry of legislation. I am interested to know whether any particular part of that is of great interest or impact. If it is, you could write to the committee and suggest where to look. I have looked at some of it, and some of it does not seem to be very impactful. If you have a handle on that, I would be grateful.
George Balston: I would briefly say that it is not where I would want it to be in terms of its comprehensiveness, but it is a good start, particularly in Illinois with SB 315 and in New York with the RAISE Act. Those are two pieces of legislation that I would consider, but, yes, I am very happy to follow up in writing.
The Chair: We have touched quite a lot on security threats, and we are going to focus the rest of the session on that, so first on cyber and quantum crypt. Liam Byrne is going to lead us into that.
Q13 Liam Byrne: You will have seen the news today. Jaguar Land Rover has announced 4,000 redundancies. It ascribes some of the costs that it is having to ameliorate to the cost of the cyberattack last year. That probably cost about £260 million in direct costs. The cost to the wider economy was about £1.9 billion, with five weeks’ worth of production shutdown. It was one of several attacks last year, which included attacks on Marks & Spencer and the Co-op.
I am interested in how you think the balance of power is going to shift between cybercriminals and UK plc. Professor Payne, do you want to kick off? Could you give us a sense of how you think AI-related threats and quantum-related threats could fundamentally change the balance of terror between cybercriminals and the mainstream economy?
Professor Kenneth Payne: Yes. I am less qualified to speak on the quantum threats, except that every quantum expert I talk to is incredibly worried about it, so I would ask Dr Cuthbert for his views on that.
Sitting suspended.
The Chair: Order. We are quorate again. Welcome back. Mr Byrne, you were pursuing a point about cyber and Professor Payne was just about to answer. Do you just want to quickly recap?
Liam Byrne: Professor Payne, you were commenting on how the balance of terror between cybercriminals and UK plc might be about to change fundamentally.
Professor Kenneth Payne: The Hugging Face episode is a good early warning sign of where we might be going. Those closed models have, in my limited experience, pretty good guardrails. A lot of resources are going into improving those guardrails to stop people being able to use them to do nefarious activities, but no guardrail is perfect. Open-weight models, which track the frontier, as we have discussed, may come without such robust guardrails.
I suspect we will arrive at a time, in the not-too-distant future, when models that are capable of offensive cyber capability are available for relatively low cost. I see that threat not going away. If anything, I see the threat that you outlined so soberly in your question increasing as time goes on.
To my mind—again, in your closed sessions you might find a more informed answer than this—the best defence to that is to have frontier models in a defensive cyber role, which, again, is another plank of our dependence on America.
Liam Byrne: Michael Cuthbert, how do we think about quantum in the context of this risk?
Dr Michael Cuthbert: I see that there are two aspects of cybersecurity for quantum or related to quantum. One is the classical cybersecurity associated with prototype and small-scale machines. There are household names, including some of the IT majors, that are pursuing quantum computing, but there are also lots of start-ups.
One of the things that the National Quantum Computing Centre has observed in establishing our testbed programme, where we have seven commercial platforms of different modalities under one roof, is that it is not just the quantum bit that you have to pursue, but it is hardening the technology right throughout the stack. The start-ups are less mature organisations. They usually have limited resources in terms of people and investment. Creating the cybersecurity within the stack to harden that as a product or service is the first aspect.
The second aspect is how, as the technology scales, we manage access to quantum compute when it gets to a crypto-relevant scale. At the moment, access to on-premises machines or cloud resources is quite tightly controlled either through the vendor or through national labs or national institutions such as ours. How that progresses as these machines become larger over time is an open question.
The saving grace may be that, as these machines scale, they will not be like a server rack where anybody could have one in a shipping container anywhere in the world. These will be large national infrastructures, and therefore Governments will get to control who has access to a crypto-relevant quantum computer. Today, it is much too early to presume that we know exactly what these future systems will look like.
Liam Byrne: In recent years, we have seen the appetite of our adversaries, such as Russia, Iran and North Korea, to work with organised crime groups in an unholy alliance to launch attacks on British business. Is it a possibility that in the future you may get crypto-capable infrastructure among our adversaries put at the disposal of cybercriminals in order to increase their lethality?
Dr Michael Cuthbert: That is certainly possible. Rather than the cybercriminals having their own quantum computer, perhaps a greater threat is them using AI or high-performance-compute models and tools that have been initiated using a quantum computer to seed the computational task, which then means their tools become enhanced. That is a more likely threat today than them having access to the hardware itself.
Liam Byrne: In a sense, the criminals would be given assets to go to market with.
Dr Michael Cuthbert: Yes. It could be some kind of computation that has been seeded using a quantum computer to either enhance the machine learning or to simulate chemical species and the like, rather than necessarily having access to a whole quantum machine.
Liam Byrne: That threat is foreseeable now.
Dr Michael Cuthbert: Yes.
Liam Byrne: Are we thinking about our defences against that peril in the right way?
Dr Michael Cuthbert: It is not robust enough yet. The National Cyber Security Centre is leading on this more than the National Quantum Computing Centre. We are much more focused on the opportunity space within industry, but there are so many dual-use applications. The development of algorithms is the critical piece that the UK is very strong on. That is what will really unlock quantum capabilities in tandem with AI and high-performance compute. It will not be one or the other; it will be in tandem.
Liam Byrne: It sounds like we are going to need some different requirements for cybersecurity and potentially a different system for cyber insurance in the future.
Dr Michael Cuthbert: Yes. Transitioning UK industry and UK Government to take up post-quantum cryptographic standards is also a vital part of this. The National Cyber Security Centre has laid out its programme or advice for companies between now and 2035, which is broadly in line with the quantum computing roadmap towards a large scale crypto-relevant machine by 2035. In my view, it is much too late to wait until 2035, when a quantum computer is available, to have implemented the correct protocols within Cyber Essentials across wider industry.
The bottleneck to that, as you described earlier, is the readiness and availability of classical cybersecurity, never mind yet another new protocol that is not fully understood and not yet operationalised because the threat is not yet operationalised.
Liam Byrne: It is not hard to foresee a scenario where you have criminals taking down, say, several food retailers coupled with a public disinformation campaign that might induce panic buying. Very quickly, the UK has a grade A crisis on its hands.
Dr Michael Cuthbert: Yes. You could imagine scenarios across a variety of different sectors, such as energy, telecoms or food and water supply chains. All these industrial sectors are dependent on computational technologies, whether it is within their supply chain, their logistics and infrastructure or their front-line services.
The Chair: Mr Balston, do you have anything to add? I am anxious to get on to the military applications in the last few minutes.
George Balston: I will briefly add that my greatest concern on criminal activity as it pertains to cyber and AI is, to Professor Payne’s point, open-weight models. Closed-weight models generally have multiple forms of protection, including monitoring at the end. When you are talking to the model, if you say something that flags the monitor, it will refuse to answer.
Open-weight models have some built-in protections. They do not have that monitoring. For the companies that are providing access to these open-weight models, it is a selling point that they never see your data. They have zero data retention, so you do not have this ability to monitor. The protection that the open-weight models have, the alignment training, is brittle because the models are open. If you can download the model weights, you can quite easily remove those safety filters.
I expect we will see large-scale cyberattacks built on open-weight models imminently. I am surprised that such a thing has not happened so far.
The Chair: That is the headline from today’s session. Well, let us see—we are about to come on to military applications.
Q14 Sarah Champion: I am interested in the panel’s views not on terrorists using AI as a threat but on AI using AI for a threat. I read the New York Times article about Hugging Face, which is absolutely terrifying: 700 agents swarming together, working collectively, having a hierarchical chain of command when they did their attack and, perhaps most terrifyingly, being sneaky and finding ways to cover their tracks. The headline of that article is basically that we are more than 50% of the way there to a full blown takeover by AI. Does the UK have the necessary understanding, protocols and protections in place to protect us in such a scenario?
Professor Kenneth Payne: The one-word answer is no. I do not think it does because this is such a new threat. OpenAI did not have it and Hugging Face did not have it. I am not sure that large sections of UK society have it.
My own area of research interest is the personas or the psychology of agents. For me, it is very close to home to see them interacting at scale and having these discussions. There is a very interesting piece that came out since that attack about sovereign AI. I cannot remember the author, but I will send it to the committee. After all, this gathering of AI was not instructed to do what it did. It went off and did it on its own. The article imagines sovereign clusters of AI doing their own thing, having escaped the sandpits and constraints that they are in and behaving as their own actors on the international stage, almost like Barbary pirates.
That is a cyber threat that we have not yet begun to get to grips with. The one thing that these models need is tokens to keep themselves running. They have to sit somewhere, even in cyberspace. If you need to pay the bills, crime pays. They are potentially very capable criminals.
Sarah Champion: Could I ask one more follow-on, Professor Payne? It goes back to an earlier question. The majority of the originators of AI are men, probably of a certain demographic. Is there a gendered aspect to the way that they are working together?
Professor Kenneth Payne: That is an active empirical question. Could I write to you with an answer to that?
Sarah Champion: I would be fascinated. Thank you so much.
The Chair: I want to move us on to the military because we have only a few minutes left and it is an important part of our inquiry. We will need to be fairly brisk through this. Lord Godson and Lord Watts are going to lead us into this.
Q15 Lord Godson: Thank you, Lord Chairman. Just very quickly, how is AI changing battlefield decision-making? This is to all three of you. What is your opinion on that, and then on the level of human control required for lethal weapons and, correlated to that, on the legal regimes that are presently in place? Can you give us a very broad overview?
The Chair: Professor Payne, this is one of your areas of expertise.
Professor Kenneth Payne: The changes, in my view, have been profound. There is a degree of debate about how profound they are, but Russia-Ukraine and the US and Israeli actions with respect to Iran have demonstrated what AI-enabled battlefield management can achieve. This is everything from planning strikes to working back through the logistic chain, developing courses of action and so on. I still feel that, for all the changes that have happened, we are in the foothills of where we are going. I expect more to come. What was the second part of your question?
Lord Godson: It was about the level of human control on lethality and the broader legal regimes.
Professor Kenneth Payne: When I started, the then chairman of BAE was on the record as saying, “We are not going to have fully autonomous weapons systems”. The discussion was about having a human in the loop. The MoD maintained that there was a distinction between automatic weapons, which were okay, and autonomous weapons, which were so far in the future as to almost be laughable. That was less than a decade ago.
I have seen the steady retreat away from human in the loop to human on the loop to meaningful human control to now through-life responsibility for designing systems. At each step, the human agency fights a rearguard action and then moves back to the next step, with the pace of the technology.
I am quite convinced that we still have a road to travel here, but meaningful human control, other than in the higher direction of war, is going to be extremely challenging when you are dealing with very large arrays of autonomous platforms that are capable of making context-rich decisions. We are not at the end point. Given where we are going, we will have to have another think about what “human control” means.
Dr Michael Cuthbert: For quantum technologies, not just quantum computing, we are at a very much earlier stage. The defence applications are more at the prototyping stage. Most important are GPS denial and positioning, navigation and timing using quantum technologies. That is not necessarily quantum computing but quantum sensing.
There is also magnetic radiometry, understanding the electromagnetic environment; brain imaging or baselining for military personnel through the course of either stressful situations or injury; hidden infrastructure and void detection to find tunnels and other hidden infrastructure underground; and anomaly detection in general, whether that is hypersonic missiles in the sky or wreckage or underground signals from submarines in the ocean. There are a number of areas, but they are still very much at the proof-of-concept or demonstrator level. The likes of DSTL and the Navy are very much engaged in that, but it is still at a prototyping stage.
Q16 Lord Watts: Just to follow up on some of that, as we have discussed, this is going at a record pace and it changes every week, but we do not seem to have made any progress at all in controlling or putting into place rules and regulations for it internationally. As I understand it, China is prepared to do that, but America is not at this point in time. How much of a threat is the fact that the present incumbent of the United States does not want to have any control outside of America?
George Balston: It is concerning. It comes back to a point that we discussed earlier around the incentives for organisations to come to the table. Frontier AI at the moment in the US is controlled by private companies, but, under the legislation that is being proposed, in certain scenarios, if there are risks that cannot be mitigated, the federal Government could step in. We can also assume that part of the AI development agenda in China is shaped by the Government there too.
Getting to a position where the frontier of AI is being moved forward by two Governments may be what it takes to bring parties to the table, because having a commercial incentive to increase the capability of your models, which is resulting in these weekly releases of new technology, is not conducive to international agreements and transparency.
Lord Watts: Do we know what that regulation would look like? Is anyone looking at, if we were to get co-operation internationally, what such a regulation system would look like, how it would work, who would control it, where it would be based and that sort of thing?
George Balston: Yes. There are people who have given thought to this. I can certainly follow up in more detail in writing, but, in summary, it would likely be based outside of the US and China in a third country that was able to verify commitments.
The commitments might relate to things such as training runs. Training runs are very expensive and take a long time. They are quite conspicuous. You have to have huge numbers of GPUs and data centres to do them. Having the ability to monitor those would be a very strong move towards international verification. You might be able to attest to the fact that you were not doing a training run, and then you might need an agreement to unlock the ability to do a future training run to build an even more powerful model.
As I say, we are nowhere near that today, but the pace of AI is moving so quickly that these are absolutely the right things that we should be thinking about.
The Chair: Thank you very much. With that, unless committee members have any further questions, we will conclude today’s session. Can I thank all members of the panel for a fascinating session, for your candour and for the depth of the answers? We really appreciate it.
Each of you has volunteered to write on one thing or another. The clerks will follow up with you. They always take a note of those things. Thank you for that. If there is anything that, on reflection, you feel it would be helpful to share with us that you did not have the opportunity to set out, please include that with anything that you are sending in.
With that, we will conclude today’s session. Thank you to the panel and to colleagues for attending.