Written evidence submitted by Atlantic Council's Democracy + Tech Initiative (SMH0034)

• To what extent do the business models of social media companies, search engines and others encourage the spread of harmful content, and contribute to wider social harms?

ANSWER: Repeated empirical evidence and internal company documents have demonstrated that social media companies’ content algorithms are designed to maximize engagement on these platforms. While there has been a push, both by politicians and those within the companies, to shift this policy to allow greater choice for users in how they view content on their feeds, metrics around engagements — including how long people stay on these platforms, how much they like, comment and share content, and whether they return to these platforms repeatedly — is still an overwhelming diving force for how these algorithms are developed. The key goal remains to keep people on these platforms for as long as possible so that companies can maximize revenue via their existing advertising models. Internal company documents and my own research demonstrate that, on average, polarizing and harmful content generally performs better, in terms of overall engagement, than less polarizing/harmful material. When you combine the dual commercial imperatives of both keeping people on the platforms and generating as much revenue via advertising from social media users, the outcome is, typically, the over-amplification of polarizing, and sometimes harmful, content by automated algorithms that often run counter to platforms’ existing terms of service dedicated to mitigate such outcomes.

• How do social media companies and search engines use algorithms to rank content, how does this reflect their business models, and how does it play into the spread of misinformation, disinformation and harmful content?

ANSWER: There is not one overriding algorithm that dictates what people see within their social media firms. In practice, overlapping and often contradictory systems have been created, over time, that optimize for different outcomes and different scenarios. As a basic principle, none of these algorithms are tailored to promoting misinformation, disinformation or harmful content. But given the overriding advertising-based business models of these platforms — and how such potentially harmful content performs better, based on engagement — there is a knock on effect in terms of automated algorithms, without the cultural or real-world knowledge to detect often legal, but harmful, material, promoting misinformation, disinformation and harmful material that runs counter to platforms’ terms of service.
 

• What role do generative artificial intelligence (AI) and large language models (LLMs) play in the creation and spread of misinformation, disinformation and harmful content?

ANSWER: The role that generative AI and large language models currently play in the creation and spread of misinformation, disinformation and harmful content is relatively small. There are many factors for this, as I outline in this 10-part series on that topic (see here https://www.politico.eu/politico-editorial-series-bots-and-ballots-artificial-intelligence-reshape-election-2024/). Currently, the are less tech-savvy means of creating such harmful content, including via existing AI tools like those within Photoshop, that do not require generative AI or large language models. The technology will inevitably improve over time. But at the end of 2024, the technology is not central to how misinformation, disinformation and harmful content is produced.

• What role did social media algorithms play in the riots that took place in the UK in summer 2024?

ANSWER: This is an impossible question to answer, given the lack of transparency and accountability in how social media algorithms currently operate. Anecdotally, content (often harmful material) related to the related to the riots gained prominence in the UK during the summer of 2024. Much of this content, though, was traditional media content of the evolving situation, and only a small percentage of the overall social media content could be deemed as harmful or illegal. To understand the impact of social media algorithms, greater access to platform data via mandatory independent data access, a report on which will be produced by Ofcom by the summer, is an urgent priority.

• How effective is the UK's regulatory and legislative framework on tackling these issues?

ANSWER: The UK’s existing regulatory and legislative framework already has many of the powers required to tackle these problems. Illegality, either online or offline, should be treated in the same way. Calls for violence on social media, for instance, are equivalent to calls for violence in the real world. What is missing, though, is a real-time response mechanism to hybrid offline-online harms that could be used in times of emergencies, including during the riots of the summer of 2024. What is important, though, is to demarcate between explicitly illegal acts (like calls for violence via social media) and harmful, but legal, acts (like the sharing of racist content via social media.) Currently, there is no regulatory or legislative mechanism sufficient to meet those needs.

• How effective will the Online Safety Act be in combatting harmful social media content?

ANSWER: At its core, the Online Safety Act is a systemic approach to improving online safety over the long-term. It is not designed to be a real-time response to often politically sensitive material being spread on social media. Where the Online Safety Act can help is by improving the transparency and accountability for social media platforms via a better regulatory and outsider research understanding of how platform algorithms potentially surface harmful (but often legal) content.

• What more should be done to combat potentially harmful social media and AI content?

ANSWER: Currently, British regulators and politicians do not have a clear understanding of the threat, or lack of one, resulting from social media and AI-based algorithms. All policymaking in this area, both in the UK and overseas, is based on anecdotal evidence and a lack of understanding over how social media companies operate internally. What is urgently needed, as stated above, is a mandatory social media data access regime — something that is already available under the European Union’s Digital Services Act — to allow regulators and outside researchers a better understanding of what happens within the companies. By prioritizing such data access, UK politicians can then be in a better place to effect meaningful change via a more comprehensive understanding over how these platforms operate. Until that happens, UK politicians will be unaware of what legislative measures are required to protect British citizens.

• What role do Ofcom and the National Security Online Information Team play in preventing the spread of harmful and false content online

ANSWER: Ofcom should not be in a position of preventing specific pieces of harmful and false content from spreading online. Its powers, under the Online Safety Act, are systemic, and any efforts to steer Ofcom’s work into handling specific issues, including large areas of potential harm like the 2024 summer riots, would risk reducing the regulator’s long-term impact. It should also be remembered that much of the “harmful and false content online” is not illegal, and individuals’ right to free speech should be preserved. Where Ofcom can have a meaningful impact is by using its regulatory powers to force social media companies to change their internal algorithms so that these companies’ actions via recommender systems meet their existing obligations under both UK law and their own terms of service.

• Which bodies should be held accountable for the spread of misinformation, disinformation and harmful content as a result of social media and search engines’ use of algorithms and AI?

ANSWER: It is often too easy to point fingers at the legitimately harmful behaviour of social media companies and search engines that surface misinformation, disinformation and harmful content. They have not done enough to uphold both UK legislation and their own terms of service, although there is still a lack of outside data to hold these firms to account. But individuals that create such social media information — as well as websites created to peddle such material, often for financial benefit — are also those who must be held accountable for their actions. It must be a combination of increased transparency and accountability for companies, in how they surface such material, a level of responsibility and accountability for individuals who produce such material and, in the long-term, a massive investment in media literacy and social media understanding so that British citizens can become inoculated to often harmful content that they see online.

18 December 2024