Written evidence submitted by the Integrity Institute (SMH0054)
II Advising the UK Parliament on Misinformation and Harmful Content
The Integrity Institute is a community of over 500 integrity experts, including professionals from major platforms such as Meta, Google, TikTok, and X. Our members are leaders with extensive experience in trust and safety teams across these platforms and bring a deep understanding of the complexities of misinformation. These comments are developed on behalf of this community. The Integrity Institute is extremely grateful for the contributions of the experts listed below.
Jeff Allen, Spencer Gurley, Matt Motyl, Nick Shen, Sofia Bonilla, Luca Belli, Yee Carter, Ariel Colon, Leah Ferentinos
We advise the UK Parliament to consider all of the following resources for the ways in which business models and platform features contribute to the spread of harmful content.
● Ranked Feeds and the Harms They Pose: A look into how algorithms can amplify harmful and illegal content, understanding the spread of content online, (the lifecycle) and the standard design of ranked feeds.
● Feed Ranking and Social Harms: A Trustworthy AI Problem: How can AI/machine learning algorithms amplify harmful content? When do ranked social feeds contribute to societal harm? How can that harm be prevented?
● Metrics and Transparency: Data, datasets, and questions to create a clear comprehensive set of transparency requirements with the goal of getting valuable metrics and transparency about platform harms.
● How Social Media Amplifies Misinformation More Than Information: Our analysis of almost 600 fact-checked social media posts revealed how algorithms and other features of social media sites (i.e. re-share buttons) help false posts go viral.
● Why Is Instagram Search More Harmful Than Google Search?: Chief Research Officer Jeff Allen examines why Instagram’s decision to disable search for sensitive topics falls short of its goal and explores the underlying reasons for its lack of impact.
● Ranking and Design Transparency: The transparency we need from platforms about their ranking and recommendation systems to ensure that they are designed, operated, and governed responsibly.
● Integrity Institute and Algotransparency Amicus Brief in Gonzalez vs. Google: An amicus brief to a Supreme Court case which describes the different design choices platforms make when building recommender systems and how those choices can increase or decrease responsibility for harms that occur on online platforms.
● Misinformation Amplification Factor (MAF): Our research hub focused on tracking how misinformation performs on platforms, in order to measure (1) the extent to which platforms are amplifying misinformation, and (2) the extent to which they are creating an incentive structure that rewards lies and misinformation online. The MAF is the level of amplification of misinformation over verifiably true content.
Rather than attempt to determine which organizations should specifically be held responsible for online harms, we have tried to map out a spectrum of responsibility for online platforms. There are instances when the companies running platforms could be less responsible for the harms on their platforms, and there are instances when companies could be more responsible for harms. This consultation should provide tools and ways to measure the level of accountability for platforms and others in the spread of misinformation, rather than a threshold dictating liability or sole responsibility.
As such, we believe that every group involved in the spread of misinformation, disinformation, and harmful content can take some accountability. For example:
● Platforms could build their recommender systems to be safer.
● Users could improve their overall media literacy.
● Parents could use parental controls to create safer outcomes for their children.
● Regulators could create clearer guidelines for platform harms and compliance.
When harm occurs on a platform, the question of how do algorithmic choices–from platforms, users’ decisions via controls, or unclear regulation–increase or decrease the level of accountability? This question poses complex philosophical and legal implications. We hope to illustrate a spectrum of accountability that applies to all main actors in the social media ecosystem, rather than constructing rigid parameters around regulators, users, and platforms. In other words, we must acknowledge the fluidity of accountability across the many players within the constantly evolving social media landscape.
When assessing accountability for the spread of misinformation, disinformation, and harmful content to online platforms, there is a spectrum of services that different companies offer, which carry different levels of responsibility for any potential risks. An example spectrum of services companies could provide, in increasing levels of accountability for any harms that occur on the platform, should include:
● Hosting services or website services: Companies can offer the opportunity to host content in return for a fee.
● Limited scale messaging services: Companies can offer direct messaging or texting between a limited number of users.
● Free hosting services and connections to followers: Companies can host content directly on their services and allow people to “follow” creators uploading content to the platform.
● Content feed services: Companies can create content feeds, which allow end users to see content from the accounts on the platform that they follow.
● Ranked feed services: Companies can apply ranking to the content they show users from accounts they follow, according to company priorities.
● Content recommendation services: Companies can recommend content to users from accounts the user has not explicitly followed, or recommend accounts for the user to follow. By enabling creators and publishers to grow large audiences on the platform, they can create incentives for users, both monetary but also attention based.
● Advertising services: Companies can distribute content to users for advertising fees. This represents the financial incentives of the company itself.
● Monetization services: Companies can allow creators and publishers to receive monetary compensation for the content they share on the platform. This represents the direct financial incentive that platforms can create for users.
In considering the spectrum of services above, and deciding where responsibility for harms increases to a critical point, it is also important to consider how the companies are designing and building their platform as well as the nature of harms. The prevalence of risky, harmful, and illegal content on online platforms will never truly be zero. But companies can ensure they are designing and building their platforms responsibly to ensure that they are doing everything possible to minimize harms. There are of course types of harms that can occur through illegal content, such as CSAM and terrorist content, but also physical harms and violence that can occur through incitement to violence. Finally, new risks are continually arising. When determining the responsibility of platforms for harms, it is also important to consider if the company is sufficiently resourcing rapid response efforts to ensure they are able to properly mitigate novel harms that can quickly arise around critical societal moments.
How the companies design, operate, and govern their platforms should play a role in assessing their responsibility. Companies should ensure that the safety of their services is prioritized. An example of how this could be assessed would be through how the company measures the safety of the platform as well as the success of the platform, and how the company manages tradeoffs between the two, when they do arise. Companies should ensure that their platform is creating positive incentives for people and societies, and they should ensure that the content they are distributing through advertising and user accounts they are partnering with through monetization programs meet high standards. And companies should ensure they are working with external experts who can ensure they are aware of the issues in countries and regions where they lack expertise internally.
If companies fail to prioritize safety, fail to ensure they are good actors in the online ecosystem, or fail to ensure they are aware of all the risks where their platform has significant usage, then it will be challenging for them to be a responsible actor. This will be especially true as the services they provide become riskier.
When assigning accountability for harmful content, it is useful to break ‘harmful content’ down further into certain categories:
There is a broad spectrum of harmful content both within and beyond these topic areas. For example, even within misinformation, it can mean users loosely re-sharing low-impact, false content or users posting incendiary calls to violence based on misinformation shared by a verified or sponsored account. This is why it is critical to break down the spectrum of accountability by content areas to ensure accountability is appropriately shared when content with differing risk profiles is amplified by different components of the online ecosystem.
Accountability can increase with knowledge. If a particular risk or harm has been well known for a long period of time, that should increase responsibility held by online platforms for their role. And responsibility changes for rapidly evolving situations, as different systems are tested during crisis moments. For example, when crisis events occur, the prevalence of risky content should be somewhat higher than baseline, and the change in events will test the company's operational capacity to detect and respond to changing situations. It could be that responsibility is lessened in these situations, although it is still important for platforms to ensure their systems are designed in a way that is safe even in evolving situations.
Lasting events, like long standing conflicts, or predictable events, such as elections, warrant an increased level of accountability held by the platforms. This is because the platform’s understanding of those issues should be robust and should include both responsible design and robust operations to ensure the safety of the platform. In addition, we highlight that prevalence tooling from platforms must be encouraged since platforms that don’t measure prevalence will not be aware of the degree to which harmful content exists on the platform and, as a result, will not take appropriate accountability.
Determining accountability is certainly a challenge that we do not yet have a regulatory framework in place to make easy assessments. Other spaces do have regulatory frameworks that determine accountability for harms, so it is certainly possible to develop a framework that properly creates accountability in how online platforms are designed, operated, and governed but unlike other regulatory areas with a clear perpetrating party, when it comes to social media platforms, many different players share accountability for a single harm. For example, a user posting misinformation may not know the content is harmful. The engineering team tasked with building a recommendation system may unintentionally push harmful misinformation to certain users. Regulators may create policy or risk assessments that fail to comprehensively encapsulate the role misinformation plays in the platform ecosystem. Responsibility for harms that arise due to our online information ecosystem should, ultimately, be shared by multiple parties. However, that shouldn’t prevent regulators from developing reasonable framework for assessing when online platforms need to be held accountable.
18 December 2024