Written evidence submitted by The Alan Turing Institute (CETaS) (SMH0007)

Written evidence from The Alan Turing Institute, prepared by Samuel Stockwell, Research Associate, Centre for Emerging Technology and Security (CETaS), The Alan Turing Institute.

The Alan Turing Institute (hereafter ‘the Institute’) is the UK’s national institute for data science and artificial intelligence. Our teams working on Defence and Security challenges conduct policy and technical research on emerging technologies. Within this context, understanding and mitigating the threat of harmful or misleading content to the UK public is a high priority. We therefore welcome the opportunity to provide evidence to the Select Committee’s investigation.

What role do generative artificial intelligence (AI) and large language models (LLMs) play in the creation and spread of misinformation, disinformation and harmful content?

1.1  Generative AI and LLMs can provide benefits at different stages of the mis- and disinformation process – from creation through to dissemination and engagement.

1.2  In relation to the creation of such content, generative AI tools lower the barriers of access for users to generate deceptive material given the ease of signing up to these services. Subsequently, alongside those traditionally associated with these activities – such as domestic extremists and foreign adversaries – members of the public may unintentionally pollute the information environment by creating highly realistic but misleading content. This was a trend that CETaS observed throughout several elections in 2024, including during the UK general election.[1]

1.3  Moreover, these tools can increase the spread of harmful content through replication and personalisation benefits. Generative AI models and LLMs enable the rapid replication of output material with far fewer resources and costs involved with other methods, such as traditional image editing software. The personalisation of disinformation activities could also reach new levels of convincingness as messages can be tailored to specific contexts or demographic groups through the large datasets that LLMs are trained on.[2]

1.4  Finally, generative AI models and LLMs can cause novel risks with engagement where users may struggle in being able to discern when they are encountering fake or misleading content online. Owing to the highly realistic nature of these outputs, which can include mimicking the likeness of celebrities or politicians, it can be difficult to identify clear flaws in the content that indicate whether it is synthetic. This risks leading to indirect implications for the wider information ecosystem, where users may increasingly be less likely to trust anything they see online – complicating fact-checking efforts.[3]  

What role did social media algorithms play in the riots that took place in the UK in summer 2024?

2.1  Shortly after the Southport incident, a number of self-described ‘news’ outlets and influential users began rapidly spreading misinformation surrounding the identity of the alleged perpetrator on social media platforms.[4]

2.2  This included unsubstantiated claims that the offender was a Muslim migrant, referenced under a false name.[5] These posts quickly garnered millions of views, promoted by social media algorithms owing to how many of the accounts were ‘verified’ on platforms such as X. This meant that such posts featured more prominently on user feeds compared to other content from unverified accounts.[6]

2.3  Outside of user feeds, the false name of the attacker was also featured as a ‘Trending in the UK’ topic on X, being suggested to users under the “What’s happening” sidebar. Meanwhile on TikTok, search results for “Southport” recommended a similar headline as a suggested query. As such, some platforms further amplified misinformation to users who may not otherwise have been exposed through their in-built news aggregator features.[7]

What more should be done to combat potentially harmful social media and AI content?

3.1  As highlighted in our recent CETaS report on AI-Enabled Influence Operations: Safeguarding Future Elections, relevant organisations should adopt a systematic framework which targets different parts of the mis- and disinformation process. This involves a focus on curtailing content generation; constraining content dissemination; countering user engagement; and empowering society to expose or undermine those spreading such content. This should be a cross-sector effort, with social media platforms working collaboratively with AI companies, fact-checking organisations, academic research institutes and government departments. Dedicated funding should be made available to support these efforts.[8]

3.2  When it comes to curtailing the generation of harmful content, DSIT should establish an implementation strategy for automatically embedding provenance records in digital content produced by the UK Government at its origin. This refers to information about when a piece of content was made, by whom, and through what means. By implementing such a scheme, this will help to strengthen the authenticity of government communications against forgeries and provide users with more confidence as to what information is coming from official government sources.[9]   

3.3  In terms of constraining dissemination, Ofcom should create a new Code of Conduct aimed at systematically targeting online disinformation practices. Drawing inspiration from the EU’s Code of Practice on Disinformation, the new code should set out self-regulatory standards for different sectors on demonetising disinformation content creators; defining unpermitted manipulative behaviours associated with fake bot accounts; providing tools for empowering users against disinformation; and requiring transparent incident reporting.[10]

3.4  For countering engagement, Ofcom should also convene major UK communications app providers and the International Fact-Checking Network to design accessible and transparent fact-checking apps for UK users. These could replicate other initiatives, such as Taiwan’s LINE app, which helps users query and verify content by providing trusted alternative news sources for cross-referencing through in-built app services.[11]

3.5  Finally, to better empower society, the UK Government should ensure that the Digital Information and Smart Data Bill or other relevant future legislation include a provision for establishing a trusted research group on disinformation. This would require social media platforms to provide trusted members of the UK academic, research and civil society communities with better data access for monitoring and flagging harmful content before it spreads further. To maintain impartiality, organisations and individuals should be selected by UK Research and Innovation’s trusted research and innovation programme.[12]

3.6  All of the above measures should adopt an interdisciplinary, cross-sector approach – recognising the now-pervasive nature of the mis- and disinformation landscape. Dedicated government research funding should be made available to support these efforts, encouraging close collaboration with international partners.

13 December 2024