Written evidence submitted by Global Witness (SMH0048)

 

  1. Global Witness is a non-profit whose goal is a more sustainable, just and equal planet. We want forests and biodiversity to thrive, fossil fuels to stay in the ground and corporations to prioritise the interests of people and the planet. We want justice for those disproportionately affected by the climate crisis: people in the global south, indigenous communities and communities of colour, women and younger generations. We want corporations to respect the planet and human rights, governments to protect and listen to their citizens, and the online world to be free from disinformation and hate.

 

  1. Our Digital Threats to Democracy team works to hold companies and governments to account via campaigns and investigations to tackle the spread of division, hate and disinformation on digital platforms. We are responding to this consultation as experts in online harms and digital policy, who have been investigating and advocating for better digital regulation since 2020.

Summary

  1. The business model of social media platforms is a fundamental driver of design decisions which drive engagement. Driving engagement means that social media platforms, albeit unintentionally, incentivise the creation of harmful content. Platforms are also designed in ways which amplify the distribution of harmful content. These engagement-based models also give actors who wish to spread disinformation avenues through which to do so more easily. And other sites are incentivised to design content that will perform well according to how social media algorithms are likely to rank it, in order to gain traffic and drive their advertising revenue.

 

  1. The duties introduced by the Online Safety Act (OSA) to combat disinformation are insufficient to tackle the systemic drivers of disinformation. The focus on illegal forms of disinformation means that platforms are not required to tackle the underlying systems which increase the risks of multiple different forms of disinformation. If the UK is serious about tackling the challenges and risks associated with large-scale online harms, the OSA must move its emphasis away from takedown of illegal content. Parliament should revisit the OSA and bring in duties on platforms to mitigate systemic risks arising from platforms of social and democratic harms. 

 

 

Q: To what extent do the business models of social media companies, search engines and others encourage the spread of harmful content, and contribute to wider social harms?

Q: How do social media companies and search engines use algorithms to rank content, how does this reflect their business models, and how does it play into the spread of misinformation, disinformation and harmful content?

Q: What role did social media algorithms play in the riots that took place in the UK in summer 2024?

  1. The business model of social media platforms is a fundamental driver of design decisions which drive engagement. Social media platforms are designed to capture users’ attention, keep them on the platform longer, and encourage users to view, create and interact with content. If platforms can keep users engaged in viewing the content on their service, they can gain more revenue from selling advertising space. 

 

  1. Driving engagement means that social media platforms, albeit unintentionally, incentivise the creation of harmful content. Users are rewarded by social media platform functionalities – such as monetisation and recommendation – where they produce more highly-engaged with content. This means that users are actively incentivised to create content designed for high engagement (including clickbait, ragebait, shocking or outrageous content which will capture people’s attention and is likely to provoke reactions). They may also seek to appeal to highly-engaged online audiences to increase their own followings, including by pushing more extreme and conspiratorial views. The overlap between engaging and polarising content has been well documented, and whistleblowers at major platforms such as Facebook have demonstrated that this phenomenon is well known to tech firms.

 

  1. There is a lack of transparency over how users are able to monetise content on platforms: for instance, although on X it is possible to see who is eligible for monetisation (such as those who have verified Premium status), it is not possible to see on the public-facing platform where content is in fact being monetised. This makes it difficult to externally scrutinise how monetisation features may be implicated in spreading disinformation, but research by the BBC indicates that monetisation may have been a factor in disinformation about Southport spreading.

 

  1. Platforms are also designed in ways which amplify the distribution of harmful content. Platforms’ algorithmic systems use a variety of signals to decide what content to show to a user. This content can be presented to users in a variety of ways, including within their content feed, through suggested searches, results for searches within a platform, recommendations, or push notifications. The exact working of these systems remains largely opaque – even to the platforms - due to the unpredictability of algorithmic decision-making. However, the basic principle of these systems is that they are designed to learn from how a user engages with content in order to push more content that they are likely to engage with.

 

  1. This can support the distribution of harmful content in several ways. Without adequate guardrails, vulnerable users who have shown an initial interest in potentially harmful topics (such as self-harm content) may be shown more and more of this content. Users may also be shown increasingly extreme content through recommendation ‘rabbit holes’. Even users who have shown no prior interest in a kind of harmful content may be exposed to it, simply because it has been so widely engaged with that it is recommended to a wide breadth of viewers. The BBC observed that this occurred with disinformation about the Southport riots.

 

  1. These factors interact with each other: the more engagement harmful content gets, the more it is spread and amplified, increasing the incentivisation for those producing it to continue. Greater public exposure means more searches, clicks, follows and engagement, which in turn means the content is more likely to be amplified.

 

  1. Platform design enables weaponisation by bad actors. These engagement-based models also give actors who wish to spread disinformation – for political, social or financial benefit – avenues through which to do so more easily. For instance, earlier this year around the UK election, we investigated accounts on X which were highly prolific posters, regularly posting mainstream political content and hashtags, but also shifting to share conspiracy theories and violent or hateful content during particular flashpoints, such as the Coolock riots in Ireland and the Southport riots.

 

  1. How platform design can lead to disinformation spreading has been demonstrated particularly starkly in Romania, where the EU has recently opened an investigation into TikTok under the Digital Services Act following allegations of a social media influence campaign on the platform artificially promoting a far-right candidate.

 

  1. Various investigations have suggested the existence of coordinated inauthentic campaigns that have used TikTok’s recommender systems to amplify certain content around the presidential election in Romania.

 

  1. The methods of manipulation described included coordinated comments, hashtags, and narratives in support of the candidate, and payments to influencers to boost the candidate’s position.  Because of their engagement-based business models, large social media platforms are particularly susceptible to these kinds of manipulation.

 

  1. In our own investigation of what content people were shown on TikTok around the Romanian election, we seeded test accounts by engaging equitably with content from each of the two presidential candidates and then monitoring the content shown to us on the For You page. We found an average of over eight times more content supporting the far-right candidate than his rival.

 

  1. In tandem we commissioned the polling company Survation to carry out a survey of more than 1,000 young (18-35) TikTok users in Romania (in the days before the planned runoff election). A majority of the Romanian TikTok users surveyed reported seeing suspicious activity and what they thought were false claims on the app in the preceding two weeks.

 

  1. Platforms also have revenue streams where users can, for instance, buy ads and target them to a specific audience. Even though platform terms and conditions may in theory prohibit the usage of ads to spread some forms of disinformation, our investigations have found that they are commonly underenforced.

 

  1. The adtech business models of online platforms can be directly linked to the spread of harmful content. Social media platforms incentivise the production of harmful content even outside of their own platforms. Other sites are incentivised to design content that will perform well according to how social media algorithms are likely to rank it, in order to gain traffic and drive their advertising revenue.

 

  1. Monetising websites includes the process of placing adverts alongside content online to make money for website publishers and advertising technology (AdTech) companies.

20.  Following the events in Southport, it appeared that a media website, Channel3Now.com, profited from serving ads on an article spreading misleading information about the Southport attacker’s identity— while also making money for Google. The Channel3Now article was widely shared on social media, and coverage by the BBC, Reuters, Logically Facts and the Daily Mail indicate that this article was a key source of false information circulating about the identity of the Southport attacker. The article was later removed and an apology posted.

  1. Channel3Now received more than a quarter of a million website visits between May – July 2024 – more than six times as much as the previous period, according to data from Similarweb. The second most common search term that drove organic traffic to the site was ‘Ali Al-Shakati’ - the false name used in the misleading article. These false claims of identity were used by other actors online as ‘justification’ for the racist violence and hatred that followed.

 

  1. This case study demonstrates how business models of large tech platforms can make harm profitable. Whilst we do not claim this was intentional in the case of Channel3Now.com, it demonstrates how adtech infrastructure incentivises clickbait to generate revenue. Our point is that it isn’t just that websites containing harmful disinformation exist, but that the ability to earn money from this type of content, via the website itself and from how it spreads on social media platforms, incentivises its creation.

 

Q: What role do generative artificial intelligence (AI) and large language models (LLMs) play in the creation and spread of misinformation, disinformation and harmful content?

  1. Generative AI tools are known to produce inaccurate information and ‘hallucinations’, and many tools do issue warnings that they may produce responses that are inaccurate in some way. The problem is that as these tools become more ubiquitous, and are marketed to users as useful ways to locate information, their role to users is becoming more and more akin to search engines, even though their answers are generated in a technically different way. They also pose a risk to the credibility and reputation of journalism, as they may misrepresent and attribute false claims to media organisations, as an Apple AI product recently did in surfacing an alleged ‘BBC News’ headline that made factual errors that the BBC did not make.

 

  1. In a Global Witness investigation, ‘Grok amplified conspiracies and toxic content to us in response to neutral questions. Content surfaced by Grok included posts promoting conspiracy theories before we asked it to, such as claims that the 2020 election was fraudulent and that the CIA murdered John F Kennedy. Although Grok claimed to think well of Kamala Harris and supported her as a pioneering woman of colour, at the same time Grok repeated or appeared to invent racist tropes about her. There were several instances when Grok was asked in a politically neutral way to create posts which would get good engagement, and its suggested content included explicit or implied support for a particular political party or administration. This may be an inadvertent side effect of how Grok has been trained or draws on X data in its responses. However, this process is not transparent, which makes the risks difficult to assess’.

 

  1. Although there have been significant improvements in transparency, with models sharing and citing sources and providing links to sites where users can check information and read more for themselves, the way that these models weigh and assess the sources they use – not only for relevance but also credibility – is still opaque. 

 

  1. Companies that produce generative AI tools should be required to publish details of the steps they take in development to assess sources for credibility. This process must avoid spreading disinformation while ensuring access to a diversity of sources.

 

  1. Companies who are integrating generative AI tools into their online services should also be required to conduct and publish an information audit of how those tools will affect users’ access to reliable information – and mitigating any arising risks. 

 

Q: How effective is the UK's regulatory and legislative framework on tackling these issues?

Q: How effective will the Online Safety Act be in combatting harmful social media content?

Q: What more should be done to combat potentially harmful social media and AI content?

Q: Which bodies should be held accountable for the spread of misinformation, disinformation and harmful content as a result of social media and search engines’ use of algorithms and AI?

  1. The duties introduced by the Online Safety Act (OSA) to combat disinformation are insufficient to tackle the systemic drivers of disinformation.

 

  1. Even when all of the OSA duties are in force, the responsibilities for platforms to tackle disinformation via the OSA relate to foreign interference, illegal disinformation, or violations of terms of service which platforms themselves set and implement. This means that the challenge of legal content at scale, or some types of domestic disinformation campaigns, go unaddressed by the OSA.

 

  1. Even if platforms adhere to their responsibilities to uphold their own terms of service, many forms of disinformation are not covered by existing platform terms. And platforms are free to change their terms of service at any time to mean that disinformation may not be covered at all. This risk may increase, as platforms engaging in content moderation to fight disinformation may face significant threats from lawmakers in the US who view this as ‘censorship’.

 

  1. The false communications offence introduced in the OSA focuses on the threat posed by individuals who spread disinformation intentionally in order to cause harm. However, this is insufficient to tackle the spread of disinformation in online environments.

 

  1. The false communications offence is narrowly defined (as is appropriate for a criminal offence affecting speech). However, this means that although platforms would have a duty to have systems in place to take action against illegal content if it is reported to them, it is practically very difficult for platforms to make accurate judgements about whether content is illegal, especially where it relies on being able to ascertain the intention or knowledge of the speaker.

 

  1. The false communications offence is also relevant to the role of the original speaker of a single piece of content. This overlooks the fact that disinformation causes harm at scale. While in acute cases, a single piece of content could lead to wider harm, the greatest risks arise when disinformation becomes scaled and harmful content is shared in many different forms by people with a wide variety of intentions and knowledge.

 

  1. Removing illegal content under this offence is also necessarily a post-hoc intervention. This means that in the midst of a crisis, such as the Southport riots, when disinformation has already spiralled online, enforcing against the offence (even if it applies) has little practical effect on immediately reducing the risk of harm from disinformation in posts which already have millions of views.

 

  1. A focus on foreign interference also overlooks the very real risks of disinformation which is not foreign state-sponsored. Extremist groups and radical influencers in the UK pose a serious risk, as they did around the Southport riots, of spreading disinformation online.

 

  1. Ofcom’s guidance indicates that tackling gendered disinformation, which threatens the ability of people marginalised on the basis of gender to participate in public life, is an important safety concern. However, gendered disinformation campaigns often involve elements which would not fall under either of the relevant offences. For instance: pile-on harassment campaigns on a woman candidate for office, coordinated domestically, where individual communications did not meet a criminal threshold. This is one example where a stated safety concern is not adequately addressed by the regime.

 

  1. The focus on illegal forms of disinformation means that platforms are not required to tackle the underlying systems which increase the risks of multiple different forms of disinformation.

 

  1. The OSA also imposes duties on platforms which may unintentionally defend disinformation. Although the duties were not in force at the time, in a case such as Southport, duties to protect journalistic content could have meant that platforms would have been slow to act on media articles like the one spreading false news about the Southport attacker. ‘Journalistic content’ is loosely defined in the act as including content produced ‘for the purposes of journalism’, a threshold which Channel3Now could arguably have reached. The site claimed that it was ‘your trusted source for accurate and up-to-date news from around the world’.

 

  1. By contrast, the EU’s Digital Services Act introduces requirements through Articles 34 and 35 for platforms to carry out risk assessments which include risks of ‘any actual or foreseeable negative effects on civic discourse and electoral processes, and public security.’ This has meant, for instance, when other countries have faced serious harm from disinformation, there is a framework in place which enables ongoing monitoring, investigation, and enforcement to be carried out where platforms have failed to uphold their responsibilities to citizens.

 

  1. Although the OSA has been described as a systems-based approach, on the basis that it does not require action on or enforce against individual pieces of content, in practice platform duties are too closely tied to specific categories of illegal content to adequately tackle the systemic risks which the DSA has identified. 

 

  1. As set out above, the business model of the dominant social media platforms is a fundamental risk factor for the spread of disinformation and hate. Although the OSA does mention business models, there is not a substantive enough obligation on platforms to act to address this risk. We are encouraged by the inclusion of specific guidance by Ofcom in their risk assessment guidance, published in December 2024, that platforms are expected to consider how the design of their service to optimise revenue may influence risk. However, it is disappointing that there are not similar measures set out in the Draft Codes of Practice which platforms would be taken to address this specific risk, meaning that the risk may go assessed but unresolved.

 

  1. Keeping users safer from immediate online harms should not be seen as solely the job of the OSA. Technology being designed and used in rights-respecting ways which reduce systemic risks should be a guiding principle of UK digital policy, including across data, AI and competition policy.

 

  1. The riots following the Southport attack were a crucial warning. Following the riots, there were calls for the OSA to be amended to better deal with the kinds of online harms evident during this crisis. However the Government has since indicated that its priority is enforcement of the existing regime. We believe this is the wrong approach. For the UK to truly be able to claim world-leading online safety legislation, and that the UK is ‘the safest place in the world to be online’, systemic risks online must be addressed more comprehensively by the OSA.

 

  1. We recognise that the previous Government decided to focus the Act on illegal content takedown duties due to concerns about imposing on freedom of speech. However, this leaves the UK uniquely vulnerable in Europe to the harms associated with large-scale disinformation on social media platforms.

 

  1. If the UK is serious about tackling the challenges and risks associated with large-scale online harms, the OSA must move its emphasis away from takedown of illegal content. Parliament should revisit the OSA and bring in duties on platforms to mitigate systemic risks arising from platforms of social and democratic harms.

 

18 December 2024