Written Evidence submitted by Logically (SMH0049)

About Logically

Founded in 2017, Logically was created to tackle the challenges of misleading, harmful and rapidly changing online narratives. Our mission is evolving and expanding as the digital landscape has become more complex. Today, with offices in the UK, US, and India, Logically is a trusted partner for organisations seeking comprehensive solutions in public safety, operational continuity, media, reputation, and national security. Our work includes supporting electoral integrity for two of the world's three largest democracies and several EU Member States, as well as supporting fact-checking for Meta and TikTok.

Executive Summary

●           Social media companies rely on advertising revenue driven by user engagement. Their platform algorithms prioritise content based on user behaviour. While this can enhance user experience, it also creates "filter bubbles" that amplify existing biases and expose some vulnerable users to increasingly extremist content.
 

●           The "illusory truth" effect demonstrates that repeated exposure to information increases belief in its truth regardless of its veracity. This exposure can lead to real-world harm, including violence. The January 6th Capitol invasion saw individuals act on false narratives they were exposed to online. Logically's work has shown that the riots in Leicester in 2022 also had this dynamic behind them.
 

●           Hostile state actors, such as Russia, exploit these dynamics to sow discord and polarise societies. Their campaigns often target conspiracy communities and fringe groups. Logically's work with Coventry University and the Commission for Countering Extremism highlights how certain types of content, shared between conspiracy and extremist groups, can fuel radicalisation. This process is facilitated by the ease with which content spreads online, exposing users to increasingly extreme ideologies.
 

●           Generative AI and LLMs have significantly lowered the barriers to entry for launching online influence operations. However, their ability to automate content creation and tailor narratives to specific audiences poses a substantial threat. AI-powered sentiment analysis allows actors to quickly assess user leanings and tailor disinformation campaigns accordingly, which can lead to the creation of highly effective and targeted influence operations.
 

●           Online platforms played a key role in spreading false narratives and inciting violence during the 2024 Southport riots. The rapid dissemination of the attacker's false name highlights how the pursuit of engagement can be harmful in the wrong context. Trending topics and hashtags amplified harmful content, contributing to the spread of hate speech and incitement to violence.
 

●           Logically's published investigation showed that a bogus news website, “Channel 3NOW,” initially published the false name of the attacker. The website shares characteristics with other bogus news sites in Russian disinformation laundering networks. While Channel 3NOW retracted the false information, they were cited by Russian state media, which amplified the content. In parallel, The UK's Head of Counter Terrorism believes foreign bots “turbo-charged" the spread of harmful narratives in the riots.
 

●           A direct link between the riots and state-sponsored disinformation cannot be definitively proven to a criminal standard. However, we believe there are reasonable grounds to infer that foreign interference activity was taking place in this period, and our investigations are evidence of that. Under the Online Safety Act, this is a priority illegal harm that would, had the relevant guidelines been in place, have required platforms to take a proactive approach to identifying and removing the associated content.
 

●           Ofcom's findings, which align with Logically's investigations, indicate that social media platforms, particularly messaging services, hosted closed groups used to coordinate violence and disseminate extremist material. The OSA's focus on illegal content, rather than "harmful" content, limits its effectiveness in addressing mis- and disinformation.
 

●           While the Online Safety Act tackles foreign interference, it lacks specific provisions for combating disinformation not sponsored by a foreign state. Ofcom's draft guidance on identifying foreign interference and "false communications" requires further clarity to ensure consistent and effective enforcement.
 

●           The National Security Online Information Team's operational response to online information risks appears limited to flagging content that potentially breaches platform terms of service, leaving the final decision to remove content with the platforms themselves. In the past, the Intelligence and Security Committee has highlighted the lack of a coordinated "whole-of-government" approach to countering disinformation, particularly from state actors like China.
 

●           The scope of the OSA should be expanded to address societal harms caused by harmful narratives. Ofcom should be provided with more explicit guidance and frameworks for identifying illegal content, particularly concerning foreign interference and "false communications.
 

●           Additionally, it is essential to mandate data access for researchers and fact-checkers to effectively enhance their ability to address information integrity risks. A more coordinated "whole-of-government" approach to combating disinformation should be established, which includes a clear delineation of responsibilities and improved inter-departmental communication.

To what extent do the business models of social media companies, search engines and others encourage the spread of harmful content, and contribute to wider social harms?

How do social media companies and search engines use algorithms to rank content, how does this reflect their business models, and how does it play into the spread of misinformation, disinformation and harmful content?
 

1.        Social media platforms' business models are intrinsically linked to user engagement. Most platforms generate revenue primarily through advertising, and their ability to attract advertisers depends heavily on balancing a large and active user base with brand safety and reputation. Some have chosen to simply prioritise engagement and do away almost entirely with brand reputation and trust and safety functions. Regardless of the emphasis placed on user safety, platforms have a powerful incentive to design their algorithms and features to maximise user engagement.

2.             The symbiotic relationship between platform business models and user engagement presents a complex challenge. While social media platforms have undoubtedly revolutionised communication and information sharing, their reliance on advertising revenue and engagement-driven algorithms has created a system susceptible to manipulation and the amplification of harmful content.

3.             One of the most tragic incidents of this took place in the run-up to what amounted to genocidal activities against the Rohingya minority in Myanmar in 2017. In 2020, for instance, three years after the violence in Myanmar killed thousands of Rohingya Muslims and displaced 700,000 more, Facebook (now Meta) investigated how a video by a leading anti-Rohingya hate figure, U Wirathu, was circulating on its site.
 

4.             Their investigation revealed that over 70% of the video’s views came from “chaining” — that is, it was suggested to people who played a different video, showing what’s “up next.” Facebook users were not seeking out or searching for the video but were directed to it by the platform’s algorithms[1].
 

5.             The most serious implications of this relate to extremist content. It is essential to be clear that social media is not the root cause of radicalisation. However, the way algorithms work leads to the creation of 'filter bubbles', which amplify existing biases that people may have. If you are susceptible to extremist and/or false content, this can lead to a social media environment being created around you that leads you to consume more and more of this kind of content over time.
 

6.             In a study run between 2018-19, researchers found that if you watched a conspiracy video on YouTube, the likelihood of getting another one recommended was about 70%. Notably, this number has since been reduced, showing that this problem can be tackled if a platform prioritises doing so[2].
 

7.             Importantly, extremist recommendations are unlikely to occur if people just watch mainstream content. Research also shows that amongst YouTube's 2 billion monthly users, only about 11% of users are responsible for about 80% of such content. However, among these 11%, about 90% of such viewers were found to harbour profound feelings of racial resentment[3]. Filter bubbles mean that people predisposed to holding extremist views have them reinforced.
 

8.             At a basic cognitive level, we are all susceptible to inaccurate narratives. Social psychology tells us that regardless of whether a piece of content is true or false, a person’s belief in the truth of that claim increases the more that it is repeated and seen by them. The more viral content is seen, the more the audience believes it is true. This is known as the ‘illusory truth’ effect[4].

9.             This effect is powerful. Some people—mainly those already susceptible—do act on the narratives they are exposed to through filter bubbles, and those acts can cause physical harm. A review of the Police files of over 400 people arrested following the January 6th invasion of the Capitol found that inaccurate narratives shaped the defendants' beliefs and actions in at least half of the cases.

10.        Logically, Coventry University and the Commission for Countering Extremism have been working to explore whether certain content types can fuel radicalisation. The partnership has designed a suite of training for Prevent practitioners and Counter-Terrorism officers that explores the role of harmful narratives in radicalisation.
 

11.        Logically has also comprehensively mapped the types of conspiracist content and narratives that are cross posted between conspiracy communities and extremist groups, and the everyday discourse often shared between these two seemingly distinct communities. This work is helping to build an understanding of the pathways to radicalisation that form through the easy spread of content, exposing users to increasingly complex ideological stances.
 

12.        The route to radicalisation - and the real-world harm it causes - is changing because of the dynamic of online channels cross-fertilising ideas, ideologies and narratives. The lines between what many commentators suggest are harmless fringe interests, and radicalisation are blurring. The emergence of the digital space means that the way radical groups form and organise themselves has fundamentally changed.

13.        In the first instance, this space offers a far broader set of routes for people to access content, with social media being just one of these. As a result, tracing a clear path to radicalisation is much more challenging than it has historically been. Individuals used to join groups and become indoctrinated through interaction with those groups. Now, they join social media channels, many of which have international reach and a complex web of interactions. Radicalisation is a much less linear process than it once was.

14.        Increasingly, it is extreme right-wing ideology that is driving real-world harm, and evidence of this is clear to see. There have been six extreme right-wing-inspired terror attacks in the UK since 2015, most recently Andrew Leak in Dover, who attacked a migrant centre. In December 2022, MI5’s Annual report noted that security services had disrupted 37 late-stage plots, a third of which were inspired by right-wing ideologies.

15.        The proportion of referrals that are made to Prevent for extreme right-wing ideology has now overtaken Islamist-related radicalisation, at 25% of the total. 25% of the total domestic security caseload is also extreme right derived, along with 22% of terrorist attacks and 28% of people imprisoned for terrorist offences[5]. Some high-profile convictions, such as that of Harry Vaughan and the Oaken Hearth Group, were partly driven by extreme right-wing content accessed through digital platforms.

16.        One distinct genre of online conspiracy theories that extreme right-wing groups are leveraging to attract people towards their ideas and beliefs is often referred to as the ‘Great Reset’. Its central idea is that global elites planned and stage-managed the COVID-19 pandemic to bring about a world government. The sudden growth of the theory offers a stark warning. Logically has calculated that from a baseline of almost zero in January 2020, total online mentions of conspiracy theories associated with the Great Reset rose to over 33 million by December 2022.

17.        To most observers, the ideas behind the Great Reset are instantly dismissible, but its adherents believe they face an existential threat that must be opposed. Because of the fear driven by the underlying belief, its supporting narratives can, and are, being skewed towards far-right rhetoric. Online radicalisation is increasingly following what many call the ‘Boiled Frog' scenario. This is the gradual exposure of individuals to increasingly extreme beliefs couched in language, vocabulary, and ideals that resonate with them, which digital platforms can, in some cases, promote.

18.        Conspiracy theories like the 'Great Reset' act as a gateway into more extreme right-wing views. They generate essentially 'grievance' content that exposes people to extreme 'bridging' narratives that offer plausible explanations for societal grievances. Once someone demonstrates an affinity for this kind of content, the way that platform algorithms function can mean they are fed this repeatedly.

19.        Examples of this kind of ‘bridging narratives’ in play are well-documented. As the BBC recently reported, newspapers such as 'The Light' now have a circulation of 100,000 copies a month with a Telegram Channel comprising 18,000 followers[6]. On Telegram, users mix content on local politics and health and wellness with content from groups like Patriotic Alternative[7].
 

20.        Matt Jukes, the UK's Head of Counter Terrorism, was correct to say in the same BBC piece that there the police are “seeing evidence of conspiracy theories being interwoven with extremism" and that this “connection is very much on our radar and in our sights as investigators”[8]. Logically’s work has identified the same narratives crossing from conspiracy sites into extreme right-wing ones and vice versa. Anti-Trans or 'groomer' narratives, as well as wider suspicion of the mainstream media, are good examples of infiltration from, and alignment with, anti-minority elements that may result in more agitated demonstrations.

21.        Such protests can quickly turn violent and be supplemented by people who previously wouldn't have been drawn in were it not for the cross-posting and viral spread of content. A recent BBC survey also found that 61.5% of people who said they would have attended rallies linked to common conspiracy theories think violence can be justified at protests[9].

What role do generative artificial intelligence (AI) and large language models (LLMs) play in the creation and spread of misinformation, disinformation and harmful content?

22.        The widespread availability of generative AI means that one of the fundamental barriers to entry for launching online influence operations- or efforts to manipulate public opinion through deception-has lowered actively. There is no longer a need to build large teams to create the content that such operations require and share it online – a requirement that previously limited the scale of such operations and made them primarily the preserve of states.
 

23.        Access to appropriately skilled and well-organised human expertise will no longer constrain the capacity to execute a large-scale influence operation. AI can largely automate content production, reduce the overhead in persona creation, and generate culturally appropriate outputs—images, text, or audio—that are less prone to exhibit conspicuous signs of inauthenticity.
 

24.        Substantial growth in social media-based influence operations will have an impact, and not just in limited online forums. There is credible evidence that such operations can cause noticeable shifts in political beliefs and behaviour and increase xenophobic or discriminatory sentiment. Short-term shifts in social media activity by extremely prominent actors can also have modest but statistically detectable impacts on racially motivated violence[10]. This is particularly pertinent given that many people now consume news through social media rather than traditional news media.
 

25.        Generative AI also delivers a step change in another essential element of a typical online influence operation: the ability to tailor content by the audience in a far more effective way. This can be done through techniques such as entity/target-based sentiment analysis and stance detection, which provide a bad actor with an automated review of the tone of what a person is posting (for example, sarcasm, confusion or suspicion) and also interpret their view as negative, positive, or neutral in terms of what it is describing or the perspective it is expressing towards the concerned entities or targets[11]. Assessing a user's leanings through their actual posts online becomes much simpler and faster using AI-powered tools. This allows for a more accurate assessment of what they will be susceptible to[12].
 

26.        Today, this kind of assessment depends mainly on inferring information, such as political leanings, from other contextual data, such as groups and people that someone follows on a social media platform, by the actor behind the influence operation. AI-powered sentiment analysis bypasses or at least eases this task.
 

27.        Generative AI also provides the capability to generate and disseminate huge volumes of content not tailored to its recipients. We refer to this as ‘flooding the zone’. This is already a strategic approach that China is known to use[13]. It is essentially the fully automated generation of vast amounts of content quickly and efficiently to flood online platforms with misleading information, disinformation, and spam, leading to decreased trust, confusion, and potential harm to individuals and societies.
 

28.        The content may be of low quality or not persuasive individually. But that does not always matter. If the goal of the actor behind an influence operation is simply to generate generic mistrust and doubt, it can often be the volume of content generated and the polluted information environment this creates that leads to mistrust.
 

29.        The scope for generative AI to do this in a fully automated way was recently demonstrated through the creation of a ‘proof of concept’ system called CounterCloud. CounterCloud’s AI identifies articles by specific publications and journalists according to its operator's brief. It then automatically prompts an LLM to generate content based on the same themes and with particular perspectives.
 

30.        The system generates fake comments, images, and sound clips to accompany the article. It generates Twitter (now “X”) posts to promote the website hosting it, counter opposing tweets via trolling, or promote positive narratives about what is being said. The total cost to deliver the whole system was US$400 [14].
 

31.        We have already started to see this approach used by hostile states. A recent report identified “Election Watch” as an inauthentic English-language political news outlet targeting US audiences with content specific to US election cycles, political campaigning, polling, and more. Advertising itself as the “go-to source for everything election-related”, the website presented itself as a balanced and non-partisan source of perspectives and issues in US politics[15].
 

32.        Much of the content was wholly AI-generated. Traffic to Election Watch was, in turn, driven by the Russian-linked online influence operation known as ‘Doppelgänger’, which comprises at least 2,000 inauthentic social media accounts. There is further discussion of Dopelganger’s use of AI in the recent US Department for Justice announcement of its seizure of 32 internet domains that it used. The relevant affidavit outlines how AI is central to this network's operation. As the FBI Director, Christopher Wray, put it, “Today's announcement exposes the scope of the Russian government's influence operations and their reliance on cutting-edge AI to sow disinformation”[16].
 

33.        On May 30, 2024, OpenAI published its first report on its attempts to disrupt deceptive uses of its tools by influence operations. They outlined how, over the previous three months, they had uncovered five covert influence operations conducted by actors from Russia, Iran, China and Israel, which used OpenAI products for various tasks. Examples included “generating short comments and longer articles in a range of languages, making up names and bios for social media accounts, conducting open-source research, debugging simple code, and translating and proofreading texts”.[17]
 

34.        According to OpenAI, the influence operations generated text and images “in greater volumes, and with fewer language errors than would have been possible for the human operators alone.” China used OpenAI tools to summarise and analyse the sentiment of large social media posts and generate responses in Chinese, English, Japanese, Korean, and other languages, then posted across different platforms.

35.        In October 2024, they released a new report on the usage of their products by influence operations. In this report, they noted that their generative AI tools were being used at a specific, intermediate stage of influence operations’ activity: “after they had acquired basic tools such as internet access, email addresses and social media accounts, but before they deployed “finished” products such as social media posts or malware across the internet via a range of distribution channels.”
 

36.        In addition to the uses seen earlier by the company, including for automated creation of content in comments and social media, their tools were also increasingly used for research purposes. For instance, the Iran-affiliated CyberAv3ngers operation sought to use ChatGPT to look for commonly used industrial routers in target locations and such devices' default passwords and security settings.
 

37.        This indicates that bad actors are already seeking to use generative AI tools and LLMs in the ways that experts have been concerned about. These tools can be used across the life cycle of an influence operation—from research into vulnerabilities to the creation of harmful content to the dissemination of this content via bots or other automated mechanisms.

What role did social media algorithms play in the riots that took place in the UK in summer 2024?

38.        The events of Summer of 2024 are by no means the first time that social media platforms have played a role in encouraging and even facilitating riots that are, at least in part, attributable to ethnic tensions. In 2022, Logically found (and published) clear evidence that riots in Leicester also followed this pattern. We showed how an identifiable influence operation unfolded over six days in September 2022. The primary platform used was X (formerly Twitter), with the campaign utilising six specific hashtags to spread its message.
 

39.        Most of the posts (81%) associated with these hashtags were geotagged to India, with only 6% originating from the UK. This suggests that the campaign may have been orchestrated or heavily influenced by actors outside the UK. The content of the posts primarily focused on claims of violence targeting Hindus and their places of worship.
 

40.        This was followed by a further wave of posts inciting Muslims and Sikhs, urging them to gather at specific locations and times and to be prepared to defend women and children within their communities. This suggests an intent to escalate tensions and potentially instigate violence[18].

41.        Two years later, similar dynamics informed the onset of riots in the UK over the Summer. Logically’s initial investigation, which we felt was in the public interest to publish as the events unfolded, informed wider public commentary on this issue. For example, articles in The Times on August 1st, 5th, 8th, 9th & 12th all explicitly referred to our published investigations. These were then widely quoted across broadcast media.
 

42.        In our view, social media algorithms played a crucial role in spreading false narratives related to the Southport attack, which became a catalyst for the riots. Several examples of this dynamic occurred over several days. They highlight how the pursuit of user engagement inadvertently prioritised the spread of inaccurate or misleading information.
 

43.        In particular, the attacker's false name, initially shared by a domestic conspiracy theorist Logically identified but did not publicly name, was rapidly circulated across multiple platforms. This person was subsequently arrested under the Online Safety Act but not charged. Logically charted how algorithms designed to promote viral content inadvertently contributed to the widespread dissemination of this narrative, potentially enabling it to reach millions of users.

44.        The attacker's false name quickly became a trending/viral topic on X, meaning users became aware of it via the platform's machine-learning recommendation system even if they weren't actively searching for it. Similarly, hashtags related to the riots, often containing harmful narratives and hateful rhetoric, gained significant traction on other online platforms, further contributing to the spread of harmful narratives.
 

45.        Ofcom has published similar findings to those seen in Logically’s report, with some platforms dealing with tens of thousands of harmful posts. This content included hate speech, incitement to violence against specific racial and religious groups, and false information designed to cause harm[19].
 

46.        Our investigations align with Ofcom’s. Some social media platforms, particularly messaging services, hosted closed groups that were used to coordinate and incite violence and disorder. Some of these groups, with memberships in the thousands, disseminated material encouraging attacks on mosques and asylum accommodations. Ofcom also notes that calls for violent demonstrations targeting a mosque were observed circulating online within hours of the vigil for the victims of the Southport attack[20].

47.        Finally, Ofcom highlights how social media algorithms, designed to maximise user engagement, contributed to the spread of harmful content. Posts about the Southport attack from high-profile accounts reached millions of users, indicating the potential for algorithms to amplify divisive narratives during a crisis.
 

48.        Ofcom does not, however, outline any real view as to whether there was any deliberate amplification of these narratives by hostile foreign states, which, if it did occur, could be a priority illegal harm under the Online Safety Act that platforms should have taken a proactive approach to managing as described in paragraph 74.

49.        In our own published reports, Logically identified a bogus news website, “Channel 3NOW”, which published the supposed name of the attacker as part of an article that was later cited by Russian state media as a source and amplified. The website’s details - such as its registrant being located in Lithuania while being based out of Pakistan and its regular cycling through other bogus news branding - and the flow of information in and out of the website resembles Russian approaches around information laundering and narrative dissemination.

50.        The website issued a retraction and apology when the name was widely reported as false; this was mirrored in Russian state media. While it cannot be proven that “Channel 3NOW” was part of a more significant foreign interference effort, its use by Russian state media to increase the circulation of disinformation that increased inter-ethnic tensions in the UK meant it functioned in a way advantageous to hostile states.

51.        Beyond this, the nature of the ‘apology’ issued by this Channel was, in our view, suspect. Their spokesperson suggested that they ‘bought the channel’ based on its existing viewership. This does not stand up to scrutiny. Before its change, the site simply hosted several videos, some in Russian, of amateur motor racing. This was not a rolling news channel. There is no apparent commercial rationale for the channel's current owner, who has been named as a Pakistani national, seeking to acquire such a channel.
 

52.        Logically’s subsequent investigations have also been supported by reports by ITV News, which have shown that Channel 3 shares a common advertising account with several other 'news' websites, including two called Fox3Now and Fox7Now. Fox3Now and Fox7Now were subject to legal action last year when the American broadcaster Fox successfully fought to regain control of those web addresses on intellectual property grounds.
 

53.        Fox3Now and Channel3Now appear to share some of the same videos on social media accounts. Their websites share near-identical logos and layouts, suggesting further evidence of a link between the publishers. In the past, Fox3 has been criticised for disseminating false narratives. For example, in 2022, it was reported that a gunman was on the loose at a shopping centre in Jersey City in the United States, prompting panic and confusion.
 

54.        All this suggests that the channel is an element within a wider disinformation ‘information laundering’ network- an increasingly common tactic that Russia and other states have used to amplify narratives for their own wider objectives. This process involves crafting deceptive narratives and disseminating them, so they appear credible within mainstream media and public discourse. It starts with the creation of false or misleading stories, often through manipulating data or sensational headlines, to lend an air of legitimacy. These narratives are then spread through multiple platforms and accounts, giving the illusion of widespread support and organic discussion.
 

55.        The strategy often leverages legitimate media outlets and aligns with existing domestic narratives, particularly those supporting Russian objectives. As disinformation integrates into the broader information ecosystem, it is made to seem as if it originated from credible sources and gained organic traction. This approach intentionally obscures the original source and intent of the narrative, making it challenging to counteract effectively once it reaches mainstream audiences[21].
 

56.        The public comments of the UK’s Head of Counter Terrorism on whether there was, or was not, foreign interference occurring in the wake of the Southport Riots are of particular interest here. He is widely quoted as suggesting that ‘foreign bots’ amplified false narratives around the Southport riots and recently told journalists that:

“As we were tracking the amount of traffic, hateful traffic, during the 24-hour period across the days, we would see tremendous spikes as around midnight, bots kicked in. And we would just see the amplification, automation of that reach of those messages which were at times hateful, at times misinformation[22]”
 

57.        It should be noted that when determining if the foreign power condition is satisfied under the National Security Act, it is not necessary to establish a direct chain of attribution to a particular state. Considering this, there are arguably reasonable grounds to infer that this was foreign interference and, therefore, behaviour within the scope of the Online Safety Act to curb.

How effective will the Online Safety Act be in combatting harmful social media content?

58.        Very deliberately, the Act sets out that except in relation to content that children might encounter- it seeks to regulate content that is clearly illegal, as opposed to ‘harmful’. This means that mis- and disinformation that is not explicitly illegal is not within scope. Comments of one former Secretary of State, Sir Jeremy Wright MP, as to why this may have been the case are particularly illustrative.

59.        He is quoted as saying that its omission was partly due to confusion between the remits of different departments. The Department of Culture, Media and Sport “was told that the Cabinet Office would be taking care of all of this. “Don’t you worry your pretty little heads about it; it’ll be done elsewhere in something called the Defending Democracy agenda,’” he says. “And then I think, subsequently, it wasn’t really”[23].

60.        In Logically’s view, this was a significant omission from the Act. The Online Safety Act is based on the idea that a duty of care is applied solely to promoting safety and preventing harm at an individual level. The Act's focus on mitigating the direct harm caused by individual pieces of content to individual people risks doing nothing to address the more pervasive indirect harms caused to individuals through the wider degradation of the civic information environment. To take the anti-Covid-vaccination movement as an example, it is difficult to capture the harm caused by even the most egregious anti-Covid vaccination messages in terms of their direct harm to individuals.
 

61.        The risk of harm to individuals from inaccurate information about vaccination comes not directly from individual pieces of misinformative content but indirectly from the support such content lends to the erroneous view that COVID vaccinations – or, for that matter, other vaccinations as well – are unnecessary or harmful.

62.        However, one element of the Online Safety Act that can effectively address some of the concerns around whether the Act does or does not help to curb this kind of behaviour is the designation of 'foreign interference' as a priority offence under it. In effect, the Act creates an obligation for social media services to proactively monitor their platforms and remove content associated with the offence (outlined in the National Security Act, 2023) before users encounter it. The Act gives Ofcom the responsibility to assess whether they are doing so effectively, ensuring that there will be accountability.
 

63.        Ofcom's main challenge is determining a suitable process or methodology for this task. To accomplish this, they must establish a shared understanding of what constitutes "reasonable inference" of foreign interference—the threshold under which content is deemed to be "illegal content" under the Act—and thus be subject to the aforementioned obligations for platforms to proactively identify such campaigns and take them down. We set out their progress in doing so below.

What role do Ofcom, and the National Security Online Information Team play in preventing the spread of harmful and false content online.

The National Security Online Information Team

64.        Between 2020 and 2023, Logically was a contractor for the unit that (until late 2023) was known as the Counter Disinformation Unit (CDU). In late 2023, that unit was renamed the ‘National Security Online Information Team’ (NSOIT). In a written answer, Ministers have made clear that its remit is currently:

“[to lead] operational response to information risks to UK audiences. Its ministerially agreed remit focuses on public safety and national security risks. This includes mis- and disinformation arising from events such as the public unrest which occurred over the summer as well as ongoing risks to the UK’s core values and democratic processes from foreign states’ interference[24]”.

65.        What this kind of operational response means in practice is the subject of much debate. Our understanding is that the team may contact social media platforms directly if they encounter content that:

“poses a demonstrable risk to public health, safety or national security [and] is assessed to breach the platform’s terms of service. No action is mandated by the government and it is up to the platform to independently decide whether or not to take any action in line with their terms of service”.[25]

66.        Parliament has consistently highlighted the complex network of government departments, regulatory bodies, and security services responsible for countering the broader threat of disinformation- especially state-sponsored. As part of their evidence to the 2020 inquiry by the Intelligence and Security Committee, the Security Services emphasised that they see their role as providing secret intelligence as context for other organisations as part of a wider Government response[26].
 

67.        They do not see themselves as holding primary responsibility for actively defending the UK’s national security from hostile foreign interference. They pointed to the Department for Culture, Media and Sport (DCMS) as having the lead in this space at that time[27].

68.        DCMS, whose portfolio of responsibilities in this area has since moved to DSIT, stated that its function was largely confined to government policy regarding the use of disinformation rather than an assessment of or operations against hostile state campaigns. The Intelligence and Security Committee noted then that:

"It has been surprisingly difficult to establish who has responsibility for what. Overall, the issue of defending the UK’s democratic processes and discourse has appeared to be something of a ‘hot potato’, with no one organisation recognizing itself as having an overall lead"[28].

69.        There is some evidence that this situation has not evolved a great deal since that time. In their report on China in 2023, the same committee noted that while China poses a “whole-of-state risk” and seeks to exert influence via disinformation, the Government is “severely handicapped” by short-termism which (according to external expert witnesses) “singularly fails to deploy a ‘whole-of-government’ approach”[29].

70.        Logically's experience engaging with the Government landscape around disinformation has thrown up some interesting examples. There are various pockets of expertise and responsibilities in different Departments, with at least four having the capability for assessing, monitoring, and/or responding to disinformation activity inside and outside the UK.
 

71.        There is a clear need to map current coordination and collaboration across the UK government and selected non-governmental partners and strengthen those efforts to support a resilient response to this risk. Efforts must focus on developing a common language, understanding, strengthening networks, and removing barriers to individual and collective responsibility.
 

72.        Some evidence of this approach emerged in the wider preparations made to ensure that any disinformation threat against the recent UK General Election was effectively managed. For example, a ‘Joint Election Security Group’ was formed, which was coordinated by the Ministry of Housing and Local Government (MHCLG).
 

Ofcom: foreign interference 

73.        The Online Safety Act (OSA) gives Ofcom the role of developing codes of practice and guidance to aid platforms in fulfilling their duties. Ofcom is also tasked with providing guidance on identifying "illegal content", including content associated with "foreign interference” as defined by the National Security Act (2023).
 

74.        In line with this, Ofcom consulted on and issued ‘illegal contents’ guidance regarding all priority illegal harms, including foreign interference. Ofcom's guidance is vital for the OSA's execution, given the lack of precedent for determining many types of illegal
content within the OSA's scope, including foreign interference offences (FIOs).
 

75.        An FIO occurs when someone acting for a foreign power engages in "prohibited conduct", resulting in an "interference effect." Platforms must assess if they have "reasonable grounds to infer" that content is linked to FIO, meeting a standard of reasonableness rather than a criminal law standard. This involves evaluating all FIO elements, including mental aspects, and ensuring no applicable defence exists.
 

76.        In July 2022, when making the case for linking FIO under the NSA with the Online Safety Bill (as it then was), Ministerial Statements were made in both Houses of Parliament outlining what the expectations of Ofcom were around this offence. They noted that:

“Like other offences in scope of the Bill, [social media platforms] would have to assess whether content amounts to foreign interference. Assessment of foreign interference activity could include judgements based on patterns of behaviours and tactics used, and contextual judgments about the intended effect of the content, which may be aided by relevant knowledge of the political and geopolitical context. In particular, we would expect platforms to consider whether repeated and persistent conduct from particular users or accounts might meet the offence. To help platforms in carrying out this duty, companies will also be able to draw on Ofcom’s codes of practice and any supplementary guidance.”[30]

77.        In its draft illegal content guidance, despite the express view of Ministers, Ofcom declined to offer meaningful guidance on this point. It stated that “there is no generic example of what foreign interference looks like” and was therefore inclined not to specify what content may be illegal in this context. In its finalised guidance, citing Logically’s response, Ofcom states that its original view was incorrect and that “existing [international] frameworks and existing generic profiles may help providers identify content which could be foreign interference”[31].
 

78.        The Carnegie Endowment for International Peace has devised an analytic framework known as the “kill chain” to examine online influence operations across social media. This framework identifies ten online behaviours to determine a campaign's country of origin. Experts in addressing foreign interference use the “kill chain” to analyse and attribute foreign interference in the UK and beyond.
 

79.        Similarly, the DISARM Framework is a global initiative combatting disinformation through data sharing and coordinated action. It includes a taxonomy of tactics and techniques used by disinformation actors and those countering them. The European External Action Service (EEAS), as does NATO, utilises this state-of-the-art framework to analyse foreign information manipulation (FIMI).
 

80.        Ofcom also asserts that ‘applied at scale’, these frameworks may detect content that does not amount to a foreign interference offence. On that basis, they decline to offer any clear thresholds or standards under which this form of priority harm might be assessed as being a ‘reasonable inference’ and, therefore, within the scope of platforms’ duty to proactively identify, and remove content associated with this offence.
 

81.        While some of the tactics used for foreign interference overlap with other kinds of coordinated conduct, it is unclear why this precludes the use of such frameworks. Looking for these tactics and techniques is merely a first step. Platforms can then assess whether the activity includes indicators of other elements of the foreign interference offence, such as other illegal or benign activity. Using frameworks such as these aims to facilitate an assessment at speed and scale. Suggesting their use by platforms does not need to be accompanied by an assertion that this is all they need to do to arrive at a determination of possible foreign interference.
 

82.        Effectively, it is left to social media platforms to make this determination with little additional guidance. There are no clear requirements- beyond a limited reference to the role of ‘bots’- placed in relation to online behaviours that they should have the capability to identify to manage this priority illegal harm. Guidance on identifying elements and harms of FIOs is crucial, as such harms can be obscured and overlooked in large-scale operations. While not all elements will meet FIO criteria, many correspond to its intended harms. There are existing internationally recognised frameworks that Ofcom could leverage. It has chosen not to do so.

83.        We remain concerned that, by doing so, Ofcom is adopting an approach that is not aligned with the European Union, where social media platforms that are signatories to the Code of Practice on Disinformation have voluntarily committed to tracking and monitoring twelve forms of online behaviour that are synonymous with this kind of activity. This code will likely become binding on signatories following its transition from a Code of Practice to a Code of Conduct. If it can be done in the EU, it is not clear why it cannot be done in the UK.
 

84.        Alongside this, Ofcom has stated that technology that platforms may need to identify foreign interference is not “reasonably available to all service providers, regardless of size, capacity or access to further information”. This statement does not acknowledge the actual status of this technology’s availability. Alongside Logically, a wide range of platforms are designing and delivering this kind of technology on the open market and have been for some years. It is available to any social media platform who wishes to use it- in the same way that other trust and safety-related technology is.
 

85.        Moreover, Ofcom could always reiterate that the principle of proportionality would continue to apply when assessing the efforts of platforms. If a platform genuinely doesn’t have the resources to engage such technology, it need not face action. On the other hand, the current approach makes it easier for even platforms with the relevant resources to avoid investing in such technology, whether they build it in-house or buy it from a supplier.
 

86.        A recent report from the leading research firm, Gartner, examined 75 disinformation security startups (including Logically) that received $747 million in venture capital (VC) seed stage (or preseed) funding from January 2020 through June 2024[32]. This is a growing market, and the UK is increasingly finding a foothold. Arguably, Ofcom should initiate a more detailed review of the solutions available to any service provider within the scope of the Act.

Ofcom: False Communications
 

87.        The Online Safety Act introduced the offence of 'false communications,' criminalising the posting of knowingly false content on social media with the intent to cause "non-trivial psychological or physical harm." Ofcom is tasked with defining what constitutes illegal content under this offence. Its draft guidance clarifies that misinformation—untrue information shared in good faith—is not covered.
 

88.        As with foreign interference, it is by no means clear what “reasonable grounds to infer” that false communications are taking place might be. Ministers, during parliamentary discussions, cited examples like a “hoax bomb threat” or promoting false remedies, such as claiming "drinking bleach cures COVID-19," when the poster knows that the information is false and could cause harm. Ofcom notes that service providers will struggle to make these determinations alone, emphasising the need for clarity on what constitutes “non-trivial” psychological harm and the relevant thresholds.
 

89.        Recent events, like the arrest (but not charge) of Bernie Spofforth- who was publicly named as being the originator of the rumour that the Southport attacker was a Muslim, illustrate real challenges in applying the law. Another individual has been successfully prosecuted following his live streaming of an apparent but fabricated, chase through the streets by ‘rioters’ on August 7th, 2024, which highlighted the offence's potential scope[33].
 

90.        Despite this, Ofcom has declined, in their recently issued Illegal Contents Guidance, to offer any threshold under which content might need to be removed because there are reasonable grounds to infer that a false communications offence has occurred. This is despite, notably, a request by X Corp. to “provide clear examples of how the offence may manifest in practice”. Ofcom state “We are concerned that providing examples would lead to an approach to content which is not sufficiently nuanced”[34].
 

91.        While a degree of caution is understandable to prevent overreach and violations of free speech, a failure to provide any further guidance runs the risk of platforms taking a hands-off approach, even in more straightforward cases of harmful communications, which itself can amount to a violation of the right to freedom of speech and expression. We note that an analysis commissioned by the European Parliament states that “the right to freedom of expression [under Article 11 of the European Convention on Human Rights] does not include freedom to use manipulative practices to spread information.”[35]
 

92.        The United Nations Special Rapporteur on the Promotion and Protection of the Right to Freedom of Opinion and Expression has also held that involuntary or non-consensual manipulation of thinking processes contravenes the right to freedom of opinion.[36] Ofcom can ensure that it qualifies any examples or guidance on what may amount to “false communications” with caveats on the need to assess the context in each circumstance so as to avoid being seen as advocating censorship. 
 

93.        In our view, the above prosecution provides some guidance on what “false communications” might mean in the context of Online Safety Act compliance by social media platforms. Issuing statutory guidance or a Ministerial Statement could clarify the definition of ‘false communications or harmful domestic disinformation, aiding law enforcement, prosecutors and the courts by helping to form a clear sense of what is and is not illegal content for the purposes of the Online Safety Act and the obligations it creates to manage such content.
 

94.        We believe such a step would be necessary if the exercise that Ofcom has announced is forthcoming, namely a second set of codes of practice, is to be effective. Although this has not been announced publicly, a recent Guardian editorial noted that "A second code, due next year, will include details about how social media companies should act in a crisis such as the summer’s riots, where online rumours were a factor in provoking racist violence”.
 

95.        We are also conscious that, as part of his recently issued ‘Statement of Strategic Priorities’ for Ofcom, the Secretary of State has clearly advocated that platforms must take proactive steps to reduce the risk of illegal disinformation inciting violence towards specific individuals or groups. It would be advisable for the committee to determine what, as part of the review that Ofcom must undertake of its new regulatory regime, it will do to satisfy the request that has been made of it.

18 December 2024