Written evidence submitted by Dr Áine MacDermott (Liverpool John Moores University) (SMH0010)

Introduction

I am a Senior Lecturer specialising in Cyber Security and Digital Forensics in the School of Computer Science and Mathematics at Liverpool John Moores University (LJMU). This written evidence is based on my work at the research centre for Critical Infrastructure Computer Technology and Protection (PROTECT) at LJMU. I would be happy to appear in front of the Committee to give an oral submission, answer any questions, and provide the Committee with any more details if needed.

Scope

This written evidence focuses on the rise in misinformation and creation of fabricated media that is becoming increasingly shared on social media platforms. I pay particular attention to the role of social media echo chambers and recommender algorithms. This written evidence will address three themes identified in the call for evidence.

Executive Summary

• Online misinformation and civil unrest in the Summer 2024 riots, caused by algorithms and increasing unreliability of data on social media platforms.
• People tend to accept information online at face value without scrutinising its accuracy or verifying the source of information.
• Social media algorithms use recommendation engines that suggest new content based on your interactions. These engines assess patterns in engagement to show similar posts. If content is gaining traction (through likes and shares), the algorithms may increase its reach by making it trend in feeds or on the main page.
• Social media facilitates echo chambers, which are environments or ecosystems in which participants encounter beliefs that amplify or reinforce their preexisting values by communication and repetition inside a closed system, insulated from rebuttal, e.g. extreme views and false media.
• Social media can be (and is) deliberately used and manipulated to incite chaos by spreading misleading information or promoting violent actions, targeting certain demographics and user groups.
• More needs to be done to hold social media platforms and messaging applications accountable for their role in civil unrest and facilitating the spread of misinformation.
• The UK government should do more to raise awareness of the questionable materials published on social media platforms and with generative AI. In particular, Online Safety campaigns in Schools could be used to help them identify questionable sources and question online narratives.
• Failure to address the above issues could have dire consequences; increased risk of online radicalisation, higher risks of malicious state actors (e.g. state sponsored criminal activity) encouraging domestic unrest, and substantially increased risks of mental health problems caused by false media online.

The rise in deepfake[1] multimedia poses significant challenges for law enforcement due to the potential impact of manipulated media on cyber investigations, questions over evidence integrity, and the reliability of visual information. Deepfakes tend to be vivid and persuasive to their viewers and can cause problems with the public, raising questions over political agendas, witness testimonies, and identification of ‘fake’ media. Fraud, evidence tampering, and the production of non-consensual pornography are just some examples of serious crimes cited by Europol (2024) [1], made possible by advances in AI technology.

In the UK, there has been a rise in illegal content and disinformation spread online ‘widely and quickly’, with the Summer 2024 riots being a notable example [2, 3]. Following the attacks, the riots and civil unrest demonstrated the role that algorithmic recommendations played in driving divisive narratives in a crisis period. According to Ofcom, the response by social media companies to this content and these actions have been “uneven” [4].

Social media platforms have been using demographic information to target users with ‘extreme’ materials. Algorithms feed off user data/input and recommend more information of a similar nature; as such, inadvertently clicking on one piece of disinformation could result in a “domino effect” whereby the user is consistently recommended similar misinformative materials [5] . In particular, the social media platform X has been a repository of questionable materials due to the introduction of new features, most notably the feature that allows any user account to become ‘verified’. This means that malicious accounts and bot accounts masquerading as legitimate users can create materials/share resources, meaning that many users would take the information as a credible source when in fact there are malicious means behind it [6].

At present there are many users who do not question online sources or narratives, becoming participants in an echo chamber (without their awareness), and users who take whatever they read/view online at face value, not questioning the origin of their sources. This is leading to a rise in mental health problems for young people caused by false media online. Also, there is an increased risk of radicalisation - disinformation and misinformation, whether intentional or accidental, can quickly spread on social media, misleading the public. False reports about police actions, political decisions, or the involvement of specific groups can fuel anger and even lead to violent outcomes.

We will explore the key questions cited in this call for evidence:

1. The links between algorithms used by social media and search engines to rank content, generative AI, and the spread of harmful or false content online.

There are strong links between algorithms used by social media and search engines to rank content. AI can detect patterns in data, then learn to make predictions from those patterns. It can then learn from these outcomes to make better predictions over time. For example, the more someone interacts with a social media platform such as Instagram or TikTok (by liking posts, creating searches, scrolling on specific topics) the more targeted the data they are presented becomes. This individual user data is aggregated and used to build audience profiles, which then further personalises messages, images, and advertisements to the specific users. With that in mind, someone liking harmful or false content online will likely be recommended similar harmful/false content. This can lead to increased risk of online radicalisation by user demographics and higher risks of malicious state actors (e.g. state sponsored criminal activity) encouraging domestic unrest.

Social media algorithms use “likes” as a key engagement signal; tailoring content recommendations based on users' past interactions. The more a user likes content, the more the algorithm understands their preferences and suggests similar material, potentially creating feedback loops and reinforcing certain types of content. This system can amplify content quickly, including disinformation or divisive material, especially during crises (e.g. the Summer 2024 riots across the UK). This can have an impact on the health and wellbeing of users as the extreme content can be overwhelming and lead to a feeling of helplessness.

Some social media platforms perform screen capturing analysing of user activity and collect information on their screen based on their browsing and searching history [14]. Even while this is not relevant to the app usage, this information can then be used to generate targeted information/advertisements for the user. Similarly, many applications collect browsing data and cookie information and use this to perform more targeted advertising. AI tools can even be used to analyse ad campaigns. For example, ‘Persado’ uses hyper-personalized AI generated content in ads to boost conversion rates across LinkedIn ads, Facebook ads, and other types of advertising and content creation. This leads to highly personalized ads that create significant uplift in performance (and revenue) [7].

AI is being introduced for search engine results and labelling AI-generated headlines (for example, the “AI Overview” tool now offered by Google provides an AI generated summary of any specific query placed by a user). However, there are cases of studies involving AI-infused search engines such as Google, Microsoft, and Perplexity returning racist comments and widely debunked scientific research [8]. For example, when generating an AI Overview on Google, for some very specific queries there can be an absence of high-quality information on the web; as such, spurious/often unreliable sources can be cited.

2. It will investigate the role of these technologies in driving social harms, with a particular focus on their role in the summer 2024 riots.

Technology has had a key role in driving social harms and division, specifically in the Summer 2024 riots. The rioting was perpetuated by misinformation that spread like wildfire, on social media and via the public, as well as behind-the-scenes influencers and so-called ‘news’ sources, both of which shared fake news with millions of viewers, some with calls to action. Known far-right activists promoted and attended the riots, e.g. some of the claims had been repeated by online influencers such as Andrew Tate, who had received millions of views on his posts repeating false narratives on X. Telegram was also used to share a poster advertising the time and place for planned protests, which was shared by many users.

Prior to the purchase of X by Elon Musk, Twitter was at the forefront of combating fake news and allowed users to flag and mark posts as ‘requiring verification’. For example, many of Donald Trump's false claims of voter fraud were labelled by Twitter users as being unverified/inaccurate. Twitter also reserved the right to remove tweets - and in extreme circumstances ban users - which it did with Donald Trump after the riots in Washington 2021 [9].

Elon Musk has since removed the 'election integrity team' at X (formerly Twitter), a department that was responsible for combating the spread of misinformation online, and removed a feature that lets users self-report false political statements. A European Commission study published by TrustLab analysed content across six social media platforms – Facebook, Instagram, LinkedIn, TikTok, YouTube, and X, and revealed that X now has the highest ratio of misinformation spread across its content [10].

Feed algorithms classify users' preferences by collecting their behavioral data, thus matching users with precise and continuous information. This matching of information gradually creates a powerful driving force for group polarization, which is highly likely to lead to the formation of echo chambers. As noted prior, echo chambers are when everything on the social media feed reflects the users interests and views without exposing them to new or conflicting/differing opinions. This can make users feel that everyone thinks the same as them, re-enforcing their views and contributing to the spread of fake news on social media [11].

Recommender and content moderation algorithms are constantly being adjusted and refined. As users continue to “like” and engage with content, social media algorithms adjust their predictions, refining what is recommended over time. This feedback loop can lead users into "filter bubbles," where they are predominantly shown content that aligns with their previous behaviour, often reinforcing existing preferences or biases. The more a user likes content from specific sources or approves of sources that align with their views, the more the algorithm amplifies similar content and excludes sources of differing views.

3. The effectiveness of current and proposed regulation for these technologies, including the Online Safety Act, and what further measures might be needed.

Current UK legislation is limited in its guidance and restrictions and requires further scrutiny to reflect real world scenarios. In practice it is hard to regulate and monitor the many online social media platforms and applications in use. However, various social media companies have instigated measures to combat current problems. For example, Instagram has introduced a ‘young persons’ account where there are more security measures in place, while easier auditing can be performed by parents or responsible adults.

However, the introduction of the recent UK Online Safety Act [12] has resulted in the following positives:

• Compliance: Ofcom will support and enforce compliance of the Online Safety Act.
• Offence categories: The Online Safety Act has been a step in the right direction, specifically categorising online acts as criminal offences such as (but not limited to):
• 1) Encouraging or assisting serious self-harm
• 2) Cyberflashing
• 3) Sending false information intended to cause non-trivial harm
• 4) Threatening communications
• 5) Intimate image abuse
• 6) Epilepsy trolling

These new offences apply directly to the individuals sending them, and convictions have already been made under the Cyberflashing and Threatening communications offences [12]. Further positives of the Act include;

• Ownership of algorithms: Platforms need to be transparent about the role of algorithms in content recommendation and moderation, ensuring that they do not promote harmful content.
• Annual transparency reporting: Social media platforms are required to publish annual reports detailing their efforts to tackle harmful content and their approaches taken.

However, the UK Online Safety Act has its limitations:

• Application of the Act and clearly defining what activities and content are deemed harmful can be interpreted differently by different social media platforms.
• While progress has been made on determining ‘information sources’, in practice it can be hard to police this as most social media platforms are hosted outside of the UK.
• Enforcing the Act across borders poses significant challenges and may require international cooperation.
• The Act places significant enforcement responsibilities on Ofcom, which may face challenges in overseeing and ensuring compliance across numerous and diverse platforms.
• Increased monitoring and data collection to comply with the Act may raise privacy issues.

Summary and Solutions

More should be done to raise awareness of the questionable materials published on social media platforms via malicious actors and/or generative AI. In particular, running Online Safety campaigns in Schools clearly promoting the new legislation could be used to help users identify questionable sources and question online narratives.

Awareness needs to be made on the repercussions of these offences so that young people and children feel more supported online. Also, promoting awareness to social media companies on the consequences for them (e.g. monetary fines) associated with criminal actions on their platforms should be further encouraged.

The EU Artificial Intelligence Act (Regulation (EU) 2024/1689) [13] classifies AI according to its risk: unacceptable, high, limited, and minimal. In short, it conducts a review and audit of AI systems and applications and requires heavy compliance and documentation on how/why/for what the data is being used for. UK companies that operate in Europe will be required to adhere to this legislation.

Social media platforms must take responsibility for the dissemination of false information online. They should be held accountable for the spread of misinformation, disinformation, and harmful content as a result of social media algorithms and AI. More regulation in this area, beginning with the introduction of the Online Safety Act, will help raise awareness of the role algorithms have in recommending and tailoring content. Furthermore, repercussions should occur for individuals and news groups deliberately spreading false information.

16 December 2024

References

[1] Europol. 2024. Facing reality? Law enforcement and the challenge of deepfakes, an observatory report from the Europol Innovation Lab, Publications Office of the European Union, Luxembourg.

[2] C. Horwood. 2024. Fake News-Driven Anti-Migrant Riots and Protests in the UK. Mixed Migration Centre. Available at: https://mixedmigration.org/fake-news-populist-violence-uk-anti-migrant-riots/

[3] BBC News. 2024. 'Riots engulf Britain' and 'summer of discontent'. Available at: https://www.bbc.co.uk/news/articles/cn38852jgr2o

[4] Tom Singleton and Graham Fraser. Ofcom: Clear link between online posts and violent disorder, BBC News. Available at: https://www.bbc.co.uk/news/articles/c70w0ne4zexo#:~:text=In%20an%20open%20letter%20setting,some%20firms%20were%20%22uneven%22.

[5] D. H. Lan., & T. M. Tung. 2024. Exploring fake news awareness and trust in the age of social media among university student TikTok users. Cogent Social Sciences, 10(1). https://doi.org/10.1080/23311886.2024.2302216

[6] T. Gerken. 2024. X gives free blue ticks to its most popular users. BBC News. Available at: https://www.bbc.co.uk/news/technology-68718291

[7] M. Kaput. 2024. AI in Advertising: Everything You Need To Know. Marketing AI Institution. Available at: https://www.marketingaiinstitute.com/blog/ai-in-advertising

[8] D. Gilbert. 2024. Google, Microsoft, and perplexity are promoting scientific racism in search results, Wired. Available at: https://www.wired.com/story/google-microsoft-perplexity-scientific-racism-search-results-ai/

[9] R. Davies. 2023. Elon Musk has removed a vital feature on X – fake news could soon get a lot worse, Yahoo! Tech. Available at: https://shorturl.at/OR6qP

[10] TrustLab. 2023. Code of Practice on Disinformation: A Comparative Analysis of the Prevalence and Sources of Disinformation across Major Social Media Platforms in Poland, Slovakia, and Spain, Disinformation Code EU. Available at: https://disinfocode.eu/wp-content/uploads/2023/09/code-of-practice-on-disinformation-september-22-2023.pdf

[11] M. Cinelli, G. De Francisci Morales, A. Galeazzi, W. Quattrociocchi, M. Starnini, The echo chamber effect on social media, Proc. Natl. Acad. Sci. U.S.A. 118 (9) e2023301118. https://doi.org/10.1073/pnas.2023301118 (2021).

[12] The UK Online Safety Act. 2023. https://www.gov.uk/online-safety-act

[13] Regulation (EU) 2024/1689 of the European Parliament and Council of 13th June 2024. Official Journal of the European Union, 2024. Retrieved from: https://eur-lex.europa.eu/legal-content/EN/TXT/?uri=CELEX:32024R1689

[14] S. Mateo. 2024. These Social Media Platforms Harvest the Most Personal Data. Kiteworks. Available at: https://www.kiteworks.com/company/press-releases/these-social-media-platforms-harvest-the-most-personal-data/