Science, Innovation and Technology Committee inquiry: social media, misinformation and harmful algorithms. 

Dr Mihaela Popa-Wyatt – Lecturer in Philosophy, The University of Manchester

This submission is supported by Policy@Manchester

Executive Summary

The business models of social media platforms incentivise the spread of harmful speech (mis-information, dis-information, polarising content, hate speech). The root cause is that platforms prioritise engagement for commercial reasons.

In the "attention economy," platforms compete for users' attention to generate advertising revenue. The more time users spend engaged, the more profitable the platform becomes. This leads to algorithms that favour sensational or emotionally charged content, regardless of its veracity or acceptability, as such content tends to create engagement. Engagement is high. There are, for example, 55 million Meta users in the UK, spending an average of 23 minutes a day on the platform. A consequence of this high level of engagement is that online search and social media have come to dominate advertising spend. In 2023, 75% of UK advertising spend was placed with search engine or social media services, a total of £27Bn.

Engagement with harmful content is also significant. A 2021 study by Ofcom estimated that “between September 2018 and August 2020, the 177 false information websites in our sample attracted on average 14 million visits a month in the UK”.[1] A 2021 NYU study found that “news publishers known for putting out misinformation got six times the amount of likes, shares, and interactions on [Facebook] as did trustworthy news sources”.[2] Finally, in a 2024 Reuters’ study, a significant proportion of users worldwide say that they find it somewhat hard to distinguish real from fake news in online platforms, including TikTok (27%), X (24%), and Meta (21%).[3] The same study estimated that 70% of UK respondents expressed concern about this. Although there is no strong evidence that hate crime itself has risen in recorded crime figures and the Crime Survey for England and Wales (CSEW),[4] there is evidence that it is a small but significant part of online content. In November 2020, Meta estimated that 0.11% of exposure to content was hate speech.[5] The organization Stop Hate UK states that this means that “for every 1,000 times a piece of content is viewed on the platform, one of them will be hateful content.”

The challenge for civil society is to reduce the amount of harmful content on social media. Current regulatory regimes (looking at both the UK and more broadly) typically have three elements. These are: (i) criminalisation of some online hate speech, (ii) requirement for governance mechanisms on the part of online platforms (e.g. under the Online Safety Act (OSA) and the current Ofcom proposals), and (iii) enforceable takedown requirements. Element (i) covers individuals, elements (ii) and (iii) cover providers. Elements (ii) and (iii) are backed up by financial sanctions, but it is expected that these should be used rarely, if at all.

My research argues that these measures are ineffective, in that they have not eliminated or arguably even significantly reduced the amount of harmful online content. Instead, the proposal here is to financially incentivize platforms to reduce or eliminate the widespread dissemination harmful content by themselves. To this end, I have proposed a tax based on the OECD’s Polluter Pays principle. This principle holds that the entities responsible for a harm should bear the costs of prevention and mitigation. Specifically, I advocate for a tax that is proportional both to advertise revenues and to the proportion of harmful content disseminated by a platform. This approach differs from the current regulatory regime and would offer three advantages over it:

• Prevention not remedy: A tax incentivizes platforms to prevent dissemination of harmful content. This differs from removing harmful content only once it has been disseminated.
• Focus on reach, not speech: By focussing on the dissemination of content rather than on individual speech the proposal avoids restrictions on free speech.
• Effectiveness at scale: the criminal courts or take down notices do not scale well. Compared to take-down requirements or criminal regulation, the tax model is more efficient at scale.

The tax proposal is described in detail in the final section.

To what extent do the business models of social media companies, search engines and others encourage the spread of harmful content, and contribute to wider social harms?

At the birth of social media, democratisation of online speech was thought to be a boon. Thirty years in, speech is routinely used to create division, spread mis/disinformation, and sow mistrust in people and institutions. There are societal costs such as:

• Amplification of Harmful Speech: Platforms like Meta and YouTube rely heavily on content curation algorithms, which promote misinformation and extremist content due to their capacity to engage users. In the 2022 Brazilian Elections, this contributed to the spread of election disinformation and misleading narratives.[6]
• Incitement to Violence: The proliferation of misinformation intensified political polarization, contributing to a climate of hostility and distrust between opposing political factions. META's recommendation systems directed users towards pro-coup content and groups in the 2022 Brazil election.[7]
• Summer 2024 UK Riots[8]: AI-generated social media posts and manipulated images bolstered false claims about asylum seekers that circulated quickly and widely. Narratives were crafted to exploit anti-immigration sentiments, leveraging generative AI's ability to mimic authoritative sources. Platforms failed to flag or remove these narratives swiftly, leading to real-world violence (attacks on mosques and asylum seekers’ accommodations).

These examples demonstrate that social-media platforms do not act quickly enough to prevent dissemination, despite current regulations.

How do social media companies and search engines use algorithms to rank content, how does this reflect their business models, and how does it play into the spread of misinformation, disinformation and harmful content?

The business models of social media platforms are driven by advertising revenue. To generate profits, these platforms employ an algorithmic ranking system which is designed to maximise engaging content. This is often sensational, emotionally charged, or divisive content. The spread of this content is accelerated in one the following ways:

• Engagement Drives Harm: Social media platforms use algorithms to prioritise engagement. This leads to amplifying sensational or harmful content because it increases screen time.
• Authority is Levelled: Social media levels source credibility.5 Individuals therefore find it difficult to distinguish reliable from unreliable sources.
• Content Homogenisation Creates Echo Chambers: Curated news feeds homogenise content with a diet of “more of what you like”. This creates echo chambers.
• Ineffective Content Moderation: Algorithms may prioritize harmful or misleading content, when content moderation is not deployed in a timely manner.

What role do generative artificial intelligence (AI) and large language models (LLMs) play in the creation and spread of misinformation, disinformation and harmful content?

Generative AI and LLMs have revolutionized how content is created and consumed. Yet, they can also be exploited to create and amplify mis/disinformation and harmful content, e.g., by:

• generating deceptive media, deepfakes and bots which can mislead or deceive audiences, phishing campaigns targeting vulnerable individuals with personalized content;
• spreading fake news, harmful ideologies, conspiracy theories, or hate speech;
• flooding platforms with misleading content, which creates the illusion of widespread consensus around false narratives or polarizing views;
• scaling content production, enabling malicious actors to produce disinformation at unprecedented speed and volume.

To minimise these risks, platforms need to ensure transparency, accountability, and use AI-generated tools responsibly.

How effective is the UK's regulatory and legislative framework on tackling these issues?

• How effective will the Online Safety Act be in combatting harmful social media content?
• What more should be done to combat potentially harmful social media and AI content?

There are challenges for current regulatory measures in general and for the OSA specifically.

• Most harmful speech does not meet the threshold for sanction under the criminal law.
• Criminal sanction and take-down notices are after the fact remedies.
• Use of take-down notices or criminalisation requires limitations on freedom of speech.
• The criminal law cannot be applied at the scale required to eliminate harmful content.

The Online Safety Act: aimed to create accountability through penalties for non-compliance, and invokes a duty of care on the part of online platforms. But the main aim of the regulatory proposal (Ofcom 2024)[9] is to ensure that platforms adopt appropriate governance mechanisms. The mechanisms proposed by Ofcom are similar to those already in place in many companies and will do little to reduce the current levels of harmful speech online.

Which bodies should be held accountable for the spread of misinformation, disinformation and harmful content as a result of social media and search engines’ use of algorithms and AI?

Because platforms and search engines amplify content algorithmically, accountability should focus on platforms. The platforms in question are the main content aggregators, social media platforms and search engines.

A Tax Proposal for Reduction of Harmful Online Content

We propose a “pollution tax” or Harmful Content tax (HCT) on online search and social media platforms. This draws on the OECD’s Polluter Pays principle, which is widely applied in environmental regulation. The "Polluter Pays" principle holds that those responsible for creating harm must bear the costs of mitigating its impact. The goal of a pollution tax is not primarily to raise large revenues, but to incentivize polluters to avoid tax by reducing pollution. Residual tax revenue has the benefit of paying for the regulatory costs, rather than placing the burden on the general tax payer.

How The Tax Would Work

• The tax would be levied on online platforms with a significant number of UK users.
• The revenue basis would be the advertising revenue generated, not profits.
• The rate of the tax would be determined by an estimate of the proportion of content exposures that contain harmful content. Harmful content exposure is how many users were exposed to harmful content. Elements of the overall measure would include mis/dis-information and hate speech (e.g., Meta's 0.11% metric).[10]

How The Tax Rate Would be Calculated

The tax rate relies on estimating the amount of harmful content exposures as a proportion of all content exposures. This is technically feasible to estimate as Meta has already published estimates. Estimating the total pollution is a necessary element of any pollution tax.

We propose that the process of certifying the proportion of harmful content exposures would be carried out by certified third party providers. The analogy to use here is that of auditors preparing financial accounts for a company. Harmful content would be defined according to standards as would the auditing procedures. Content auditors would be certified against the standards.

The tax rate could be incremented in bands, or be proportional to the harmful content exposure rate (HECR). As an illustrative example of the latter, I assume a harm to tax rate coefficient of 50. If the HECR for a provider were 1 in 1000 (or 0.1%), the tax rate would be 5%. If the HECR were instead 1 in 100, the tax rate would be 50%. These rates of tax are on revenue. So if we assumed an average 10% tax rate (arising from an HECR of 1 in 500 and a coefficient of 50), the annual tax revenue (via approximating that platform providers of any size would be subject to the tax) would be of the order of £2.7Bn (using 2023 UK advertising spend with online search and social media platforms). These levels of tax are significant enough to induce the desired tax avoidance behaviour by platforms to reduce their tax by reducing their dissemination of harmful content.

Allocation of Tax Revenue

The revenue generated from the HCT would be invested to achieve societal goods, e.g., by allocating it to:

• digital literacy programs to help users identify misinformation and fake news.
• independent regulatory bodies to fund oversight and research.
• law enforcement and organizations combating hate speech and misinformation.

An Alternative

Paul Romer’s Digital Ad-Tax[11] proposal has some similar features. He proposed a progressive tax targeting revenue from digital advertising. The idea was to incentivize the companies to move away from advertising-based revenue models toward subscription models, or to split into smaller, independent firms. The tax's progressivity increases the burden on larger firms, aiming to diminish their economic and political dominance. By taxing advertising revenue directly, the plan avoids issues like income-shifting between tax jurisdictions. The difference with the proposal here is that I choose to focus on the actual harms created. The proposal described in this evidence document avoids regressively displacing advertising spend from online providers to less efficient forms of advertising.