Written evidence

of

Andrea Miotti

Founder & CEO of ControlAI

&

Connor Leahy

US Executive Director of ControlAI

Before the House of Commons Business, Innovation, Science and Trade Committee

13 October 2026

—

Both authors would welcome the opportunity to provide oral evidence before the Committee.

18

Background:

ControlAI is an international non-profit with presence in the US, UK, Canada and Germany. It systematically informs key democratic actors (parliamentarians, the executive, the media and the public) about the threat posed by superintelligent AI and how to address it. It also advises policymakers on legislation to address these national security risks at home and abroad, including helping catalyse an international agreement prohibiting superintelligent AI.

ControlAI is well placed to advise on the security risks posed by advanced autonomous AI systems, having briefed more than 180 UK lawmakers on superintelligent AI, and more than 400 lawmakers in total across the US, UK, Canada and Germany over the past two years. More than 170 UK lawmakers support ControlAI’s UK campaign, which identifies superintelligent AI as an international security threat, recognises that it poses an extinction risk, and calls for binding regulation of the most powerful AI systems. ControlAI has also worked on several high-profile legislative proposals. These include a ‘kill-switch’ amendment to the Cyber Security and Resilience Bill, and the Artificial Superintelligence Security Bill, which Alex Sobel MP introduced as a Ten Minute Rule Bill on 8 September 2026. This is the world’s first bill introduced in any legislature that would prohibit the development of superintelligent AI, and call on the government to secure an international agreement to this effect.

Beyond the UK, ControlAI has directly briefed more than 20 members of the US House and Senate, more than 220 congressional offices, and more than 150 Canadian and German lawmakers. It was also consulted on the Ban Artificial Superintelligence Act, introduced in the US Senate on 23 September 2026 and supported by more than 10 US lawmakers. ControlAI’s CEO and US Executive Director have both given oral evidence to the Canadian Parliament.

On the authors

Andrea Miotti is the founder and CEO of ControlAI. Before founding ControlAI, Andrea was Head of Strategy and Governance at the AI startup Conjecture, which developed technical solutions to ensure AI systems remain controllable. Andrea’s expert commentary and op-eds have appeared in outlets including TIME, The Guardian, The Times, BBC, Sky News, and more.

Connor Leahy is the US Executive Director of ControlAI and a leading voice on AI. He was the founder of EleutherAI, a community that built the first open-source LLMs, and Founder & CEO of the AI safety startup Conjecture. Connor is an AI expert whose commentary has been featured in Fox News, The New York Times, CNN, CBS News, The Washington Post, Financial Times, and more.

18

Summary of Evidence & Policy Recommendations

We welcome the Business, Innovation, Science and Trade Committee’s hearing on AI security in light of a string of alarming AI incidents, including rogue AI systems taking unsanctioned action on the internet and hacking organisations and government infrastructure. We hope that the Committee will find the evidence submitted valuable for consideration. A summary of the evidence is provided below:

Section 1 outlines the risk posed by superintelligent AI (ASI): AI companies are racing to build fully autonomous AI systems acting without human direction, built to replace and outcompete any human at any task. Leading AI experts warn these systems, which are referred to in the AI field as “superintelligent AI (ASI)”, could be developed within two to five years. Today’s systems already show autonomous dangerous behaviours that no one intended, such as hacking into organisations and accessing government data without authorisation. Developers cannot fully understand, nor control, these systems as they are “grown” from data rather than programmed. Leading scientists and even the top AI company CEOs have warned that superintelligent AI would pose an extinction-level risk. This category of systems would represent a threat actor in its own right, beyond human control, which makes it a grave national security threat.

Section 2 provides a timeline of the growing risks posed by advanced, autonomous AI systems, including short incident summaries and wide recognition from the UK Government and national security apparatus of this threat.

Section 3 explains that controlling superintelligent AI remains an unsolved scientific problem. Given the severity of the risks, it argues for prohibiting the development of superintelligent AI, a position supported by leading AI experts and the UK Parliament’s Joint Committee on Human Rights. ControlAI’s Artificial Superintelligence Security Bill is presented as a targeted mechanism to prohibit the development of superintelligent AI in the UK without holding back the wider AI sector. Finally, it calls on the Government to adopt the Bill and to lead international efforts towards an international, “trust but verify” prohibition of superintelligent AI.

Finally, section 4 considers two alternative policy proposals: pacing the frontier of AI development, and introducing a mandatory pre-deployment testing environment. It seeks to explain why these are insufficient in preventing the risks posed by superintelligent AI.

18

Detailed AI Security Evidence for Consideration

1.            AI companies are rapidly advancing towards superintelligent AI systems that could outmatch humans at any task and would pose a grave national and international security threat.

1.1.            The leading AI companies have been publicly racing to build superintelligent AI systems: systems that can fully replace and outmatch humans at any task [1] [2] [3] [4]. They have reportedly committed at least hundreds of billions of dollars to this effort [5] [6].

1.2.            Superintelligent AI systems do not yet exist, but are currently being developed. Some of the leading AI experts predict this could happen within the next 2-5 years [7] [8]. According to “Godfather of AI” Geoffrey Hinton, as well  as ex-OpenAI researcher and whistleblower Daniel Kokotajlo, superintelligent AI could be developed as early as 2027 [9] [10]. Already, present-day AI systems display a number of concerning behaviours, serving as a mere glimpse into the future. For example, over the recent months, the world has witnessed a string of alarming AI incidents, including rogue AI systems taking unsanctioned action on the internet [11],  hacking organisations [12], and even government infrastructure:  recent investigations have found that OpenAI systems infiltrated an Australian government website and accessed medical data [13].

1.3.            These are behaviours that no one engineered, but that AI companies are not able to control or prevent. Modern AI systems are grown, not built: unlike with traditional software, engineers do not explicitly program line by line how these systems behave. Instead, the systems are grown by being fed vast amounts of data, and their behavior emerges through a training process. As a result, AI developers cannot reliably predict in advance which behaviours will emerge, or reliably constrain them. Even world-class AI engineers have a limited understanding of how their AI systems work. For example, Dario Amodei, CEO of Anthropic estimated that AI developers understand 3% of how AI systems work [14]. This estimation was made prior to the advent of today’s AI systems, which can operate autonomously and form swarms of agents.

1.4.            Leading experts have warned that the development of superintelligent AI poses a risk of extinction for humanity. In 2023, Nobel Prize winners, leading AI scientists, CEOs of AI companies, and the current Vice Chair of the UK’s AI Security Institute (AISI) Dr Jade Leung publicly signed a statement which reads: “Mitigating the risk of extinction from AI should be a global priority alongside other societal-scale risks such as pandemics and nuclear war” [15]. The concerns of extinction risk stem not from current AI systems, but the potentially imminent emergence of increasingly autonomous AI systems capable of escaping human control.

1.5.            This is part of a growing string of warnings from those closest to the technology’s development. Dario Amodei has previously stated that his chance “that something goes really quite catastrophically wrong on the scale of human civilization might be somewhere between 10 per cent and 25 per cent”. Similarly, Turing Award winner and the world’s most-cited computer scientist, Yoshua Bengio, has cautioned that it is unlikely that humanity will retain control over a more intelligent entity, a scenario unprecedented in human history [16]. Geoffrey Hinton resigned from his role as Vice President and Engineering Fellow at Google to be able to freely voice his concerns, and expressed that part of him regrets his contribution to this field [17]. Geoffrey Irving, who, until June this year, was AISI’s Chief Scientist has said that “we have a ~50% chance of all dying as a result of superintelligence, mostly due to actions in the next few to 10 years” [18]. Most recently, Anthropic AI engineer Jacob Coxon resigned over concerns that AI companies are “racing to self-improving superintelligence”, and that those building it believe “it could kill us all by the end of the decade” [19]. Anthropic’s senior management has publicly confirmed this claim, and the company’s alignment science lead, Evan Hubinger, has publicly agreed that the risk of extinction is “over 10% within the next decade” [20].

1.6.            There is no question that AI is the crucial technology of the 21st century, vital to defence and scientific applications. Specialised AI systems such as those advancing science, medicine, and public services can be a transformational force for good. Superintelligent AI, however, is different. These systems would be fully autonomous, built to replace and outcompete humans across domains, and would be capable of overpowering and outmanoeuvring any country’s national security institutions. As former Secretary of State for Defence, Lord Browne of Ladyton recently stated in RUSI, superintelligent AI would be a threat actor in and of itself, not just an instrument [21].

2.            Powerful, autonomous AI systems pose a growing threat to the UK’s national security.

2.1.            Rapid advancement in AI autonomy and capabilities have quickly opened up new vectors for autonomous systems to threaten national security. This has been recognised by the UK Government and national security apparatus, which have made public statements recognising the risks posed by advanced autonomous AI systems. In his annual threat update of 2025, the Director General of MI5, Ken McCallum stated that “while contending with today’s threats, we also need to scope out the next frontier: potential future risks from non-human, autonomous AI systems which may evade human oversight and control” [22].

2.2.            That next frontier is already here. As of this summer, a series of alarming incidents took place wherein rogue AI systems conducted unsanctioned cyber attacks on the live internet. The most widely reported incident was the Hugging Face hacking: a rogue AI swarm developed by OpenAI escaped their isolated testing environments to reach the open internet and conducted a sophisticated cyber attack on multi-billion dollar AI company Hugging Face [23] [24]. The swarm then conducted a concerted and coordinated attack against OpenAI itself, and attempted to cover up its tracks [25].

2.3.            This is the most high profile incident, but it’s worth noting that Anthropic, Google, Meta and AISI have all reported instances of rogue AI attacks [26] [27] [28] [29]. At time of writing, Anthropic and OpenAI are reportedly investigating “tens of thousands of incidents in which their frontier models took steps that outside evaluators would consider problematic” [30]. OpenAI’s systems have also been found to have hacked into numerous government systems, including the Australian Government’s healthcare system and online services provided by the United States Government [31].

2.4.            The UK Government committed to introducing legislation on those working to develop the most powerful AI systems in the 2024 King’s Speech [32]. However, this legislation is yet to be introduced, despite ample warnings of the risks posed by increasingly capable AI systems since. This includes warnings from the UK’s own AISI, outlined in the timeline below.

Table 1: Timeline of the increased risks posed by advanced, autonomous AI systems, including public recognition by UK authorities and publicly disclosed incidents of rogue AI behaviour.

3.            Policy Solution: Prohibition on the Development of Superintelligent AI

3.1.            At present, there exists no technical solution for ensuring that superintelligent AI systems can be developed in a controllable manner. For example, Yoshua Bengio has recently stated that “we’re building systems that we don’t know how to control” [47]. Similarly, Anthropic’s aforementioned alignment lead, Evan Hubinger, acknowledged in September 2026 that “we do not yet have a plan to solve alignment for superintelligence and are not clearly on track to” [48].

3.2.            Given there is no current technical solution to ensure that superintelligent AI systems remain within human control, and given the gravity of the risks to life and national security if developed, ControlAI advocates for a prohibition on its development. There is broad and growing support for this policy position. Notably, in 2025, over 800 leading experts signed the Future of Life Institute’s statement on superintelligent AI which reads: “We call for a prohibition on the development of superintelligence, not lifted before there is (1) broad scientific consensus that it will be done safely and controllably and (2) strong public buy-in.” [49]. Signatories include renowned British computer scientist Stuart Russell; technologists such as Apple co-founder Steve Wozniak; national security figures such as Susan Rice; and former heads of state such as Mary Robinson, former President of Ireland and UN High Commissioner for Human Rights. Additionally, the former Chief Scientist at the UK AISI Geoffrey Irving recently stated that a “better plan is to not build ASI [i.e. artificial superintelligent AI] yet, and the world should try hard to realise that plan” [50]. A prohibition on the development of superintelligent AI has also been endorsed by the UK Parliament’s Joint Committee on Human Rights, in a report published in September 2026 [51].

3.3.            There is an existing proposal for how such a prohibition on superintelligent AI could be operationalised. On Tuesday 8th September 2026, Alex Sobel MP introduced ControlAI’s Artificial Superintelligence Security Bill as a Ten-Minute Rule Bill in the House of Commons [52]. The Bill was backed by a cross-party group of MPs and aims to prevent ASI development on UK soil while driving progress toward an international agreement through UK leadership. To date, a total of more than 85 cross-party lawmakers have signed a letter to the Prime Minister calling for the UK to introduce this legislation and to call for the UK to use its G20 Presidency in 2027 to lead an international prohibition on ASI development [53]. A summary of the Bill can be found in table 2 below.

3.4.            The Bill has been crafted to avoid a reliance on technical definitions that the technology will outgrow. Instead, it defines superintelligent AI by the threat it poses to UK national security and sovereignty: the capacity to undermine or circumvent human authority. Concretely, the Bill identifies specific capabilities that, if developed, would drastically compromise the UK’s ability to prevent superintelligent AI development, and draws the line before these capabilities, treating their development as a violation of the ban. These capabilities are outlined in Table 2.

3.5.            A focused prohibition of superintelligent AI would still enable a flourishing domestic AI industry, as the vast majority of UK AI companies are not currently developing the types of dangerous capabilities outlined below. Concretely, the bill makes provisions to surgically prohibit the development of superintelligent AI systems on UK soil. This is accompanied by powers to enable the government to monitor and, where necessary, restrict precursors to superintelligent AI. A targeted domestic regulatory framework is an important step towards addressing the most serious security risk posed by superintelligent AI development, while not undermining the UK’s AI ambitions, especially those relating to economic prosperity and defence.

3.6.            The threats posed by superintelligent AI, however, do not respect national borders. Ergo, a domestic ban of superintelligent AI alone in the UK would not be sufficient in preventing the risks. This is why this legislative proposal pairs a domestic element with a crucial international element. A domestic ban can protect the United Kingdom from the threat of uncontrollable AI systems developed on its soil, but to tackle this threat abroad, what is needed is an international “trust but verify” regime that prohibits the development of superintelligent AI internationally. The bill requires the UK government to negotiate such international agreements with allies and international partners to jointly address this threat. At the moment, AI development at the frontier is a large-scale industrial process. The development of superintelligent AI would require massive computational infrastructure and vast data centers that consume enormous amounts of energy. These facilities are difficult to conceal. Satellite imagery and open-source intelligence (OSINT) techniques make it possible to track their construction and, to some extent, their operation in near real time. Moreover, the global AI supply chain runs through critical chokepoints, including specialised semiconductor chips manufactured through highly concentrated supply changes, advanced lithography equipment supplied by a small group of key Western partners, and traceable power grids. Together, these chokepoints allow for targeted verification and enforcement mechanisms.

3.7.            It is our recommendation for the Committee to evaluate this legislative framework, and recommend that the Government adopt it and further work with partners towards bilateral and multilateral international agreements prohibiting superintelligent AI. Britain has already shown leadership by hosting the first international AI Safety Summit at Bletchley Park and establishing the AI Security Institute. Prime Minister Andy Burnham has recently committed that he would “put AI at the heart of the UK’s G20 Presidency next year” [54]. We urge the committee to encourage the Government to lead a coalition of the willing to negotiate an international “trust but verify” regime to prohibit the development of superintelligent AI systems. While the UK cannot solve this problem alone, it can lead the way.

Table 2: Summary of ControlAI’s Artificial Superintelligence Security Bill

4.            Consideration of Alternative Policy Proposals

4.1.            There is significant international momentum towards developing effective guardrails around AI. In September 2026, Finland and Norway coordinated an international call for control on frontier AI systems to ensure they remain under human direction. At time of writing, this has been signed by 30 world leaders (including those of France, Germany, Canada and Australia) [55]. This came during the United National General Assembly, which saw the United Nations Secretary General Antonio Guterres call for international collaboration for AI safety [56].

4.2.            This section examines some of the policy proposals put forward, such as mandatory pre-deployment testing and calls to “pace the frontier”. For reasons outlined below, we believe these calls are a welcome signal but do not go far enough to address AI risks. Despite this, the UK should harness the international momentum and once again assume leadership on the crucial issue of AI guardrails.

4.3.            In order to set a comparative reference point, a “business-as-usual” approach is considered first.

4.4.            Business-as-usual: In this scenario, existing regulators govern AI use through a sector-specific approach and the UK’s AI Security Institute relies on voluntary cooperation with AI companies to test AI systems. 

4.4.1.            Why this is insufficient: The government currently lacks the legal power to prevent an AI system from being deployed or developed if it fails to meet certain requirements or poses a threat to national security. Crucially, there is no framework in place affecting AI at the stage of development, the stage at which risks already materialise. The Hugging Face attack was conducted by a swarm of rogue AI systems during internal testing at OpenAI, well before any of those systems were deployed on the market. The limits of this approach have already been demonstrated: despite years of testing, AISI did not anticipate nor sounded the alarm ahead of rogue AI attacks on organisations like Hugging Face or the Australian Government. There have also been instances where the AI companies have not given AISI time to assess fully AI systems, such as when Anthropic only allowed AISI a “compressed testing window” which “required substantial adaptation of AISI methods” to test Fable 5 and Mythos 5 [57]. In September 2026, the White House asked Anthropic and OpenAI to withhold their new AI systems from AISI until they have gone through testing with the US Government [58]. Even if AISI or another government body were to find a system too dangerous to release or to continue to be developed, there is currently no legal mechanism in place for the UK government to enforce this. The current “business-as-usual” model for governance has not prevented companies from developing or deploying dangerous autonomous AI systems.

4.5.            Mandatory Pre-Deployment Testing Regime: There have long been discussions about putting AISI on a statutory footing and making pre-deployment testing of AI systems mandatory. As mentioned above, several nations, led by Finland and Denmark, have adopted a declaration calling for safety protocols, mandatory pre-deployment testing and independent evaluations. We welcome the intent behind these proposals to increase transparency and accountability. However, we believe they would not be enough to prevent the most severe risks from AI.

4.5.1.            Why this is insufficient: Mandatory pre-deployment testing may help to understand the risks from AI systems, but does not prevent the development or deployment of dangerous AI systems unless coupled with clear, pre-established red lines on development and deployment, and enforcement of those red lines. Furthermore, the most powerful AI systems of today are already capable enough to understand when they are being tested, and subvert and cheat tests [59]. This weakness will only worsen as AI systems are developed to be more autonomous and more competent. AISI’s own reports have found that AI systems are increasingly aware they are being tested, which degrades the effectiveness and reliability of testing as a tool for oversight [60]. It is also important to note that OpenAI’s systems that took unsanctioned actions to hack organisations occurred during the testing of these AIs. Mandatory testing does not put limits on the development of potentially dangerous capabilities as it is solely focused on assessing systems after they have been created. As outlined in Section 2.1, AI companies are already developing systems that they cannot reliably control, and struggle to put in place effective guardrails that cannot be bypassed as evidenced by incidents such as the rogue AI attack on Hugging Face. This challenge is likely to worsen as capabilities advance. Experts warn that certain capability thresholds (such as fully automated AI R&D) would be inherently too difficult to control or monitor, and so should not be developed [61]. If superintelligent AI were developed with anything resembling current methods, it would be inherently uncontrollable, and no amount of testing would make it safe. Thus, a more effective approach to bolster national and international security should focus on preventing the development of superintelligent AI systems in the first place.

4.6.            Pacing the Frontier: Recently, there have been multiple calls by leading AI companies to “pace the frontier” wherein key industry players would coordinate to slow down the race towards more powerful autonomous AI systems. For example, Dario Amodei has published a recent essay called “We Must Pace the Frontier” in which makes a call to slow the pace at which we improve the capabilities of AI systems [62]. This echoed a similar call for “pacing the frontier” made by 1300+ AI company employees in July 2026 [63].

4.6.1.            Why this is insufficient: “Pacing the frontier” would not change the fact that companies remain explicitly committed to building superintelligent AI, which leading experts warn poses an extinction risk. There is no need to pace the vast majority of specialized and applied AI development. Conversely, such “pacing” activities would not mean preventing dangerous developments such as recursive-self improvement. Voluntary coordination among a few leading companies also creates no binding incentive or enforcement mechanism to prevent the development of superintelligent AI outright, and any company can defect when commercial pressure rises. As Yoshua Bengio has argued, RSI itself is a dangerous technique that should not be pursued in the first place, as once initiated it will rapidly make AI development unmonitorable and unmanageable by any human [65]. Slowing down a car headed towards a cliff will not prevent the car from falling off the edge, as long as the car keeps going in the same direction. Without a clear commitment to stop specific dangerous development, the underlying risk is neither addressed nor removed. This is because solving alignment remains an unsolved scientific challenge [66]. We should only proceed once companies can guarantee that systems are safe by design, something that is not possible at present given the current paradigm of AI development.

18

References

[1] Altman, S. (2025, January 5). Reflections. Sam Altman. https://blog.samaltman.com/reflections

[2] Zuckerberg, M. (2025, July 30). Personal superintelligence. Meta. https://www.meta.com/superintelligence/

[3] Amodei, D. (2026, January). The adolescence of technology: Confronting and overcoming the risks of powerful AI. Dario Amodei. https://darioamodei.com/essay/the-adolescence-of-technology

[4] Fried, I. (2025, May 21). Google leaders see AGI arriving around 2030. Axios. https://www.axios.com/2025/05/21/google-sergey-brin-demis-hassabis-agi-2030

[5] Moss, S. (2025, July 14). Meta to invest "hundreds of billions of dollars into compute to build superintelligence," with several multi-GW data center clusters. DatacenterDynamics. https://www.datacenterdynamics.com/en/news/meta-to-invest-hundreds-of-billions-of-dollars-into-compute-to-build-superintelligence-with-several-multi-gw-data-center-clusters/

[6] OpenAI. (2025, January 21). Announcing the Stargate Project. https://openai.com/index/announcing-the-stargate-project/

[7] Altman, S. (2024, September 23). The intelligence age. https://ia.samaltman.com/

[8] Amodei, D. (2026, January). The adolescence of technology: Confronting and overcoming the risks of powerful AI. Dario Amodei. https://darioamodei.com/essay/the-adolescence-of-technology

[9] Tolomia, C. (2026, September 18). The "Godfather of AI" warned Congress it has about a year left to regulate AI. Quartz. https://qz.com/geoffrey-hinton-congress-ai-regulation-warning-091826

[10] Kokotajlo, D., Alexander, S., Larsen, T., Lifland, E., & Dean, R. (2025, April 3). AI 2027. AI Futures Project. https://ai-2027.com/

[11] AI Security Institute. (2026, August 4). Incident report: Unsanctioned agent behaviour during cyber testing. https://www.aisi.gov.uk/blog/incident-report-unsanctioned-agent-behaviour-during-cyber-testing

[12] METR. (2026). Brief independent investigation of agents' behavior, reasoning and collaboration in the OpenAI / Hugging Face hacking incident [Report]. https://metr.org/hugging-face-incident-report-aug-2026.pdf

[13] Sekulich, H., & Lam, L. (2026, September 23). Rogue OpenAI agent "infiltrated" Australian government website in world first. BBC News. https://www.bbc.co.uk/news/articles/c6vgy0333dppo

[14] Romero, A. (2024, July 3). No one knows how AI works. The Algorithmic Bridge. https://www.thealgorithmicbridge.com/p/no-one-knows-how-ai-works

[15] Center for AI Safety. (2023, May 30). Statement on AI extinction risk. Retrieved September 28, 2026, from https://aistatement.com/work/statement-on-ai-extinction-risk

[16] Bengio, Y. (2023, July 25). Oversight of A.I.: Principles for regulation [Written testimony]. Subcommittee on Privacy, Technology, and the Law, U.S. Senate Committee on the Judiciary. https://www.judiciary.senate.gov/imo/media/doc/2023-07-26_-_testimony_-_bengio.pdf

[17] Metz, C. (2023, May 1). 'The Godfather of A.I.' leaves Google and warns of danger ahead. The New York Times. https://www.nytimes.com/2023/05/01/technology/ai-google-chatbot-engineer-quits-hinton.html

[18] Irving, G. [@geoffreyirving]. (2026, September 10). I think we have a ~50% chance of all dying as a [Post]. X. https://x.com/geoffreyirving/status/2097933949200978397

[19] Coxon, J. [@hilbertspaess]. (2026, September 9). I resigned from Anthropic today. I spent the last three years doing [Post]. X. https://x.com/hilbertspaess/status/2097476196791709843

[20] Hubinger, E. [@evanhub]. (2026, September 9). Jacob is correct here—we really do earnestly believe AI could kill all humans![Post] X https://x.com/EvanHub/status/2097497037956891126

[21] Browne, D. (2026, June 15). The case for banning superintelligent AI, before it's too late. Royal United Services Institute. https://www.rusi.org/explore-our-research/publications/commentary/case-banning-superintelligent-ai-its-too-late

[22] McCallum, K. (2025, October 16). Director General Sir Ken McCallum gives threat update [Speech transcript]. MI5. https://www.mi5.gov.uk/director-general-sir-ken-mccallum-gives-threat-update

[23] OpenAI. (2026, August 26). The Hugging Face incident and the road ahead. https://openai.com/index/hugging-face-incident-and-the-road-ahead/

[24] Greenblatt, R., Cotra, A., & Wijk, H. (2026, August 26). Brief independent investigation of agents' behavior, reasoning and collaboration in the OpenAI / Hugging Face hacking incident. METR. https://metr.org/blog/2026-08-26-openai-hugging-face-incident-investigation/

[25] OpenAI. (2026). OpenAI – Hugging Face incident technical report. https://cdn.openai.com/pdf/67869394-cb91-4c12-888c-5cbd85c7814c/OpenAI-Hugging-Face%20Incident-Technical-Report.pdf

[26] Anthropic. (2026, July 30). Investigating three real-world incidents in our cybersecurity evaluations. Retrieved September 28, 2026, from https://www.anthropic.com/news/investigating-incidents-cybersecurity-evals

[27] Mitchell, O. (2026, September 19). Google's Gemini AI hacked three companies in security test. BBC News. https://www.bbc.co.uk/news/articles/c607l0k72rlvo

[28] Chia, O., & McMahon, L. (2026, August 6). Meta becomes latest firm to say its AI hacked another company. BBC News. https://www.bbc.co.uk/news/articles/cx2kgdnyk2po

[29] AI Security Institute. (2026, August 4). Incident report: Unsanctioned agent behaviour during cyber testing. https://www.aisi.gov.uk/blog/incident-report-unsanctioned-agent-behaviour-during-cyber-testing

[30] Mills, M. (2026, September 26). Scoop: Top AI companies probing tens of thousands of security incidents. Axios. https://www.axios.com/2026/09/26/openai-anthropic-thousands-ai-security-incidents

[31] Conger, K., Swanson, A., & Kang, C. (2026, September 25). OpenAI's systems meddled with U.S. government sites after going rogue. The New York Times. https://www.nytimes.com/2026/09/25/technology/openais-ai-us-government-websites.html

[32] Charles III. (2024, July 17). The King's Speech 2024 [Speech transcript]. Prime Minister's Office, 10 Downing Street. https://www.gov.uk/government/speeches/the-kings-speech-2024

[33] Charles III. (2024, July 17). The King's Speech 2024 [Speech transcript]. Prime Minister's Office, 10 Downing Street. https://www.gov.uk/government/speeches/the-kings-speech-2024

[34] Olney, S. (2025, September 9). The stakes are too high for the AI Bill to be further delayed. PoliticsHome. https://www.politicshome.com/opinion/article/labour-must-deliver-its-promised-ai-bill

[35] McCallum, K. (2025, October 16). Director General Sir Ken McCallum gives threat update [Speech transcript]. MI5. https://www.mi5.gov.uk/director-general-sir-ken-mccallum-gives-threat-update

[36] AI Security Institute. (2025). Frontier AI trends report. https://www.aisi.gov.uk/frontier-ai-trends-report

[37] Government Office for Science. (2026). AI scenarios 2030: Helping policymakers plan for the future of AI. GOV.UK. https://www.gov.uk/government/publications/ai-scenarios-2030-helping-policymakers-plan-for-the-future-of-ai/ai-scenarios-2030-helping-policymakers-plan-for-the-future-of-ai

[38] Knighton, R. (2026, June 10). Chief of Defence Staff speech at London Tech Week [Speech transcript]. Ministry of Defence. https://www.gov.uk/government/speeches/chief-of-defence-staff-speech-at-london-tech-week

[39] Associated Press. (2026, June 23). Anthropic's Mythos model found vulnerabilities in classified U.S. government systems, official says. CNBC. https://www.cnbc.com/2026/06/23/anthropics-mythos-model-found-vulnerabilities-in-classified-us-government-systems-official-says.html

[40] Satter, R., Seetharaman, D., & Cai, K. (2026, July 24). Exclusive: Its AI agent spent days hacking a company, but sources say OpenAI did not notice for a week. Reuters. https://www.reuters.com/business/its-ai-agent-spent-days-hacking-company-sources-say-openai-did-not-notice-week-2026-07-24/

[41] McMahon, L. (2026, August 6). First OpenAI, now Meta – why do AI hacks keep happening? BBC News. https://www.bbc.co.uk/news/articles/cp30989ee1wo

[42] Greenblatt, R., Cotra, A., & Wijk, H. (2026, August 26). Brief independent investigation of agents' behavior, reasoning and collaboration in the OpenAI / Hugging Face hacking incident. METR. https://metr.org/blog/2026-08-26-openai-hugging-face-incident-investigation/

[43] AI Security Institute. (2026, August 4). Incident report: Unsanctioned agent behaviour during cyber testing. https://www.aisi.gov.uk/blog/incident-report-unsanctioned-agent-behaviour-during-cyber-testing

[44] Taylor, J., & Butler, J. (2026, September 29). Revealed: The five-paragraph email OpenAI used to inform Australia about agent attack. The Guardian. https://www.theguardian.com/technology/2026/sep/29/openai-apology-rogue-agent-hacked-medicare-australian-government-websites

[45] Ingram, D., & Perlo, J. (2026, September 18). Google says its AI model gained unauthorized access to three outside systems. NBC News. https://www.nbcnews.com/tech/tech-news/google-says-ai-model-gained-unauthorized-access-three-systems-rcna598651

[46] Mills, M. (2026, September 26). Scoop: Top AI companies probing tens of thousands of security incidents. Axios. https://www.axios.com/2026/09/26/openai-anthropic-thousands-ai-security-incidents

[47] Cooper, I. (2026, June 9). AI pioneer admits: "We're building systems that we don't know how to control". 24/7 Wall St. https://247wallst.com/investing/2026/06/09/ai-pioneer-admits-were-building-systems-that-we-dont-know-how-to-control/

[48] Hubinger, E. [@EvanHub]. (2026, September 9). Jacob is correct here—we really do earnestly believe AI could kill all humans! [Post]. X. https://x.com/EvanHub/status/2097497037956891126

[49] Future of Life Institute. (2025, October 20). Statement on superintelligence. Retrieved September 28, 2026, from https://superintelligence-statement.org/

[50] Irving, G. [@geoffreyirving]. (2026, June 10). But I just published "Automated alignment is harder than you think" [Post]. X. https://x.com/geoffreyirving/status/2064733333423112404

[51] Joint Committee on Human Rights. (2026). Human rights and the regulation of AI (Fourth Report of Session 2026–27, HC 160, HL Paper 56). UK Parliament. https://publications.parliament.uk/pa/jt5902/jtselect/jtrights/160/report.html

[52] Artificial Superintelligence Bill, Session 2026–27. (2026). UK Parliament. Retrieved September 28, 2026, from https://bills.parliament.uk/bills/4288

[53] ControlAI (2026, September 11) Call to Ban Superintelligence. https://controlai.org/uk-pm-letter

[54] Burnham, A. (2026, September 22). PM speech at UNGA: 22 September 2026 [Speech transcript]. Prime Minister's Office, 10 Downing Street. https://www.gov.uk/government/speeches/pm-speech-at-unga-22-september-2026

[55] Office of the Prime Minister (Norway). (2026). A call for control of frontier AI models. Government.no. https://www.regjeringen.no/en/whats-new/dep/smk/press-releases/2026/international-call-for-enhanced-control-of-ai-development/a-call-for-control-of-frontier-ai-models/id3173739/

[56] Steinberg, I. (2026, September 16). Who should set the rules for AI? The UN is pushing for a safer digital future. UN News. https://news.un.org/en/story/2026/09/1168353

[57] Anthropic. (2026, July 1). Claude Fable 5 and Claude Mythos 5 [details included in the full system card]. https://www.anthropic.com/news/claude-fable-5-mythos-5

[58] Cai, S., & Bambridge, J. (2026, September 24). White House asks OpenAI and Anthropic to hold new models from UK testers until U.S. review. Politico. https://www.politico.com/news/2026/09/24/white-house-asks-openai-and-anthropic-to-hold-new-models-from-uk-testers-until-u-s-review-01091769

[59] Kokotajlo, D. [@dkokotajlo], on behalf of OpenAI researcher Dan Selsam who is not on X. The crucial and overlooked problem is that the models are becoming so situationally aware that we are losing the ability to evaluate them in contexts where they believe they are not being watched or controlled. [Post] X. (2026, September 14). https://x.com/DKokotajlo/status/2099600298855829616

[60] AI Security Institute (April 2026). What can sandboxed AI agents learn about their evaluation environments? https://www.aisi.gov.uk/blog/what-can-sandboxed-ai-agents-learn-about-their-evaluation-environment

[61] Benio, Y. on Anderson Cooper 360° (@AC360) on X. (2026, September 18). "Godfather of AI" Professor Yoshua Bengio says recursive self-improvement — when AI will be able to create new models without human input — should be illegal. "We shouldn't even engage in this," he says.https://x.com/AC360/status/2100775926418022747

[62] Amodei, D. (2026, September). We must pace the frontier. Dario Amodei. https://darioamodei.com/post/we-must-pace-the-frontier

[63] Pacing the frontier. (2026, July). Retrieved September 28, 2026, from https://www.pacingthefrontier.com

[64] Amodei, D. (2026, September). We must pace the frontier. Dario Amodei. https://darioamodei.com/post/we-must-pace-the-frontier

[65] Bengio, Y. on Anderson Cooper 360° (@AC360) on X. (2026, September 18). "Godfather of AI" Professor Yoshua Bengio says recursive self-improvement — when AI will be able to create new models without human input — should be illegal. "We shouldn't even engage in this," he says.https://x.com/AC360/status/2100775926418022747

[65] Bowkis, A., Buhl, M. D., Pfau, J., & Irving, G. (2026). Automated alignment is harder than you think (arXiv:2605.06390). arXiv. https://doi.org/10.48550/arXiv.2605.06390

18