Written evidence submitted by Beatriz Kira, Zoe Asser, Phoebe Li and Julie Weeds (SMH0056)

 

University of Sussex

 

Evidence from academics at the University of Sussex to the Science, Innovation and Technology Committee’s inquiry into social media, misinformation and algorithms

 

 

We welcome the opportunity to submit evidence to this Science, Innovation and Technology Committee’s inquiry into social media, misinformation, and the role of algorithms. This submission is authored by law and computer science scholars from the Sussex Centre for Law and Technology (SCLT) and SussexAI at the University of Sussex. Leveraging our academic expertise in law and technology, our analysis focus on the functioning and risks of recommendation algorithms on social media platforms.

Drawing on evidence of the role social media platforms played in the Southport events last summer (Q3), we examine how these algorithms influence the spread of online content and their associated risks (Q1, Q2). We scrutinise the UK’s legal and regulatory framework, including the Online Safety Act 2023 (Q4) and explore additional approaches to enhance accountability over these algorithms (Q5).

Based on our analysis, we propose recommendations to mitigate the spread of misinformation, disinformation, and other forms of harmful content facilitated by social media’s use of recommendation algorithms.

 

I.           Summary or arguments and recommendations

  1. Modern social media platforms have transformed into complex systems of communication and information intermediation, where recommendation algorithms are a cornerstone of their engagement-driven business models. Central to their business model is the monetisation of user attention, driving a constant pursuit of increased user engagement. To do so, these platforms use sophisticated recommendation algorithms that leverage user data to target content and curate personalised feeds. This algorithmic control over information flow grants platforms significant influence over public discourse, yet the role of these algorithms remains largely under-scrutinised.
  2. While limited transparency into the functioning of recommendation algorithms hinders a full understanding of their impact, evidence suggests that their optimisation for engagement contributes to the amplification of harmful content and makes users more vulnerable to falsehoods. The risks of harm extend beyond illegal content (such as hate speech and false communication) to content that falls below the threshold of illegality (including abusive speech targeting race, religion or gender). In addition to the harm caused by the content itself, research shows that recommendation algorithms can create filter bubbles and echo chambers, distorting public discourse, limiting diversity of viewpoints, and eroding shared understandings of reality. This in turn can make people more susceptible to believing in falsehoods.
  3. In contrast to analogous European regulation, the UK Online Safety Act 2023 (OSA) provides no safety duties that focus on the development and adoption of recommendation algorithms by platforms, nor measures that could have prevented or mitigated the spread of different types of harmful content during events such as the Southport riots. The OSA’s duties largely focus on illegal content and content harmful to children, which has significantly limited its ability to counter misinformation and disinformation outside the narrow legal definition of the new “false communications offence” and other forms of harmful but lawful content. Much of the distressing content circulated during the Southport riots, which contributed to the violence, would not constitute illegal material and therefore would not be subject to platforms’ risk assessments and safety duties, raising concerns for adult users. Most platforms would not only lack legal obligations to moderate such content but could also potentially exacerbate its spread due to engagement-driven systems.
  4. The most relevant provisions in the OSA for tackling the abusive speech, rumours, and falsehoods prevalent during the Southport riots are those applicable to Category 1 platforms—including the duty to enforce terms of service consistently and transparency reporting obligations. However, these provisions are yet to be implemented. Even when implemented, they will not apply to all relevant platforms. The Secretary of State for the Department for Science, Innovation and Technology (DSIT) has set categorisation thresholds that exclude smaller but riskier platforms from Category 1. This exclusion disregards advice from civil society and ignores research demonstrating that smaller, fringe platforms were instrumental in spreading harmful, extremist content related to the riots (ISD, 2024). By excluding these platforms from Category 1, the OSA effectively exempts them from the crucial requirements, leaving a significant gap in the Act’s coverage.
  5. To enhance transparency and enable effective regulation of social media recommendation algorithms, legal and policy changes are necessary. Most social media companies are not forthcoming about the functioning of their algorithms, relying on legal protections such as intellectual property law to shield their systems from scrutiny. To better understand the impact of these algorithms on the dissemination of content online, particularly misinformation and disinformation, and design appropriate regulatory responses, researchers, regulators and policymakers need to be able to scrutinise what recommendation algorithms are trained to do, and what inputs they use – i.e., to understand why a certain piece of content was recommended to a user, or why certain content tends to be amplified.
  6. Based on our analysis, we suggest the Committee consider the following recommendations (further detailed in section III below):

6.1. Ofcom should include a “break-the-glass” mechanism in its online safety codes of practice. This mechanism would require services within the scope of the Online Safety Act to respond swiftly to unfolding events by taking urgent measures, including adjusting their recommendation algorithms.

6.2. Ofcom should leverage its investigatory powers and the transparency notices for Category 1 services to require companies to increase the accountability of recommendation algorithms. This should allow the regulator to demand technical access to companies’ recommendation algorithms for auditing purposes, including via an API.

6.3. The government should introduce legislation placing legal duties on social media platforms to assess and mitigate the safety risks posed by their recommendation algorithms. These duties should prioritise algorithmic functionalities over specific content categories.

6.4. Future AI regulation should require that recommendation algorithms deployed by social media companies undergo scrutiny by external auditors. Importantly, this should include not only auditors hired by the platforms but also independent audits by public interest researchers.

6.5. Platforms should carve out exceptions in their policies and lawmakers and policymakers should include safe harbour provisions in legislation to allow public interest researchers to conduct safety assessments of recommendation algorithms.

 

II.        Detailed analysis

Social media’s engagement-based business model and the role of recommendation algorithms

  1. Social media platforms rely on user engagement to drive their business models. Like all business, social media companies can only exist if they make money. Yet, the average user does not pay to be a member of a social media network or to receive or generate content—due to a cross-subsidy model made possible by the multi-sided market structure of platforms. There are many different forms of advertising which take place on social media platforms, but they are all more effective with more users spending more time on the platform. Therefore, whilst companies do not generally share the details of the algorithms that they use to promote or recommend content, we can be confident that the guiding principle is one of increasing engagement.
  2. Modern platforms are designed with features that go beyond basic hosting and transmission, embedding algorithms that personalise what each user sees to drive such engagement. Platforms prioritise content based on various metrics—such as click-through rates, user retention and ad revenue (Pattrn.AI)—effectively curating content by amplifying certain elements while demoting others (Keller, 2021). If a platform can predict that a user will be more likely to engage with a post in some way (e.g. liking or resharing it) then this is a better post to recommend to that user. This is true whether the post is paid for by a company advertising their services or whether it is a free post by another user. Targeting specific adverts at users means they are more likely to respond positively, which makes advertising more effective without increasing the overall volume of adverts.  Recommending other user posts which a user is more likely to engage with increases the amount of time the user spends on the platform, making it more likely that they will engage with future targeted advertising, and also makes many of the popular recommendation algorithms more accurate by feeding them with more data.
  3. While the workings of these algorithms are often opaque, in many platforms they are designed to reward content most likely to elicit user interaction. In the simplest case, one could just recommend more highly the posts which have the most engagement from other users. However, it would take time for posts to gain traction, and it may be difficult for new, previously unseen posts to ever be recommended by the algorithm. Behavioural data, such as past engagement patterns, is therefore crucial for recommendation engines. The more data a platform has, the better it can predict future user behaviour, and whether or not people are likely to engage with it. How many followers does the user have and how popular are their recent posts? This would still lead to all users being recommended much of the same content. Of course, content can also be recommended based on who a user follows. However, the guiding principle of increasing engagement means that social media platforms want to expand people’s networks which means they want to show them content from other users who they are likely to then choose to follow.
  4. One way recommendation algorithms drive engagement is by measuring user similarity based on followers and content interactions. For two similar users, it then makes sense, if a platform is seeking to increase engagement, to promote content to one which the other has engaged with. User A usually likes the same things as user B, user B has liked this new post / follows user C, therefore show content from user C to user A (collaborative filtering). We can take two things away from this. First, posts will be promoted if they are predicted as likely to receive a lot of user engagement, which leads to a feedback loop with this content being amplified on the platform and reaching more users. Second, different users will have different posts promoted to them based on their preferences and past behaviour on the platform. While this allows users to find relevant information and build social relationships, it also creates risks (as we will discuss in the next section).
  5. Emotionally charged content is more likely to be amplified. Research suggests that platforms prioritise implicit feedback content that is likely to elicit strong emotional responses, as this generates high engagement (Narayanan, 2023). By optimising for engagement, these platforms inadvertently amplify content that appeals to our unconscious, automatic, emotional responses. This feedback loop further strengthens existing trends, making it more likely that emotionally charged content will go viral compared to more balanced and nuanced information (Narayanan, 2023).

What are the risks involved in social medias’ recommendation algorithms?

  1. While the lack of transparency surrounding social media recommendation algorithms makes it difficult to understand their impact on the spread of content, research has identified two primary concerns. First, these algorithms can directly amplify the presence of harmful content if it drives engagement. Second, by creating echo chambers and filter bubbles, these algorithms can distort public debate, limiting exposure to a diversity of viewpoints and eroding shared understandings of reality.
  2. On the first point, by prioritising content that is likely to generate high engagement, these algorithms can contribute to the spread of falsehoods and other forms of harmful content. A key issue is the amplification of content that, while not explicitly illegal, can still be harmful in high-doses, when disseminated widely (Howard & Kira, 2023). For instance, resharing of rumours and unfounded claims about the Southport assailant, while not necessarily meeting the legal threshold for illegality, contributed to the escalation of tensions and fuelled the riots. To be sure, a relevant chunk of the content that circulated like wildfire was illegal content and therefore should have been taken down by platforms’ content moderation responses (as in the case of racial slurs or hate speech)—as we discuss below. The fact that it wasn’t removed highlights the failures and limitations of such systems. However, even if every piece of illegal content had been taken down, the flood of anti-immigration content and the rumours that took over social media during the riots themselves would have also caused significant harm. The amplification of harmful, yet legal, content can have, and has had, significant consequences.
  3. On the second point, recommendation algorithms can distort the structure of public debate, shielding users from being exposed to diverse viewpoints and making them more vulnerable to falsehoods. By serving users only with content they are likely to engage with, rather than content that challenges their views or interests, users may become unaware of alternative perspectives, only seeing posts that confirm their existing viewpoint. Controversial posts are only likely to be shared with those users who will agree with the viewpoint of the poster. These mechanisms—often referred to as filter bubbles, or echo-chambers—limit the average user’s exposure to a plurality of views and the quality of information accessed. Users within that bubble become more convinced that everyone agrees with them, while users outside of the bubble may not know it exists or how many people are inside it. While limited empirical evidence exists on the extent or harmfulness of filter bubbles, research indicates that it may increase users’ exposure to misinformation (Rhodes, 2021) and make them less critical of it (Tommasel & Menczer, 2022), with implications for the likelihood of believing in falsehoods and contributing to their further dissemination.

How effective is existing legislation in tackling these risks?

  1. The Online Safety Act 2023—currently being implemented by Ofcom—is the main instrument available to regulate social media in the UK, but it offers limited means to provide a timely and effective response to counter the role of recommendation algorithms in the dissemination of disinformation and misinformation. The legislation places a series of duties of care on in-scope services—user-to-user services (such as social media platforms and messaging apps) and search engines. These duties cover illegal content (covering existing criminal offences and new offences introduced by the OSA) and content harmful to children. Larger and riskier platforms that meet the Category 1 thresholds face additional duties, including user empowerment duties. The fact that the OSA’s main risk assessment and mitigation duties for adults are focused on illegal content means the law offers limited levers to require companies to effectively address the risks posed by content that doesn’t cross the illegality threshold (Judson, Kira & Howard, 2024). This is problematic because, as argued above, and shown in the case of the Southport riots, lawful content when amplified by platforms can still lead to real-world harm.
  2. The primary legal provision introduced by the OSA to address disinformation, the false communication offense, targets only a very specific type of harmful conduct. The new “false communications offense” (Section 179) makes it a crime the sending of a message with information that one knows to be false with the intent to cause “non-trivial psychological or physical harm” to a foreseeable audience, without a reasonable justification (Section 179). The offence is committed even when harm is intended towards a general group rather than identified individuals. However, the requirement that the sender know the information is false significantly limits its scope. In the dissemination of falsehoods and rumours, such as during the Southport riots, only posts shared by individuals who knew the information to be false could be considered illegal content—which likely wasn’t the case for many individuals resharing the content.
  3. For illegal content, such as content that constitutes the new false communication offense, service providers are subject to duties of care. They are required to assess the risk of users encountering such content (Section 9, OSA) and to effectively mitigate and manage the risks of harm to individuals (Section 10(2), OSA). They must also implement systems for the swift removal of illegal content upon receiving reports or discovering it independently (Section 10(3), OSA). Services’ terms of service must explicitly outline user protection measures for illegal content, and platforms are held accountable for consistently enforcing these terms (Sections 10(5), 10(6), OSA). The effectiveness of these duties will become clearer when Ofcom’s guidance and codes of practice come into force in 2025. However, based on the draft documents published by the regulator, the focus for platforms will be on the analysis and take-down of specific pieces of content, rather than the implementation of systemic measures to address infringing content on a larger scale, and respond to wider crisis in a timely fashion (Judson, Kira & Howard, 2024).
  4. User-to-user services are required to introduce proportionate measures relating to algorithms to mitigate and manage risks, but Ofcom’s recommended approach insufficiently addresses recommender systems. All user-to-user services must introduce proportionate measures relating to the design of functionalities and algorithms to mitigate and manage the risk of harm of illegal content, which would include recommender systems (Section 10(4), OSA). However, there is only one measure relating to recommendation algorithms in Ofcom’s code of practice on illegal content. This requires user-to-user services to carry out on-platform tests to collect safety metrics that would allow them to determine if design adjustments to their recommender systems increase exposure to illegal content (Ofcom, 2024). While it is important that services are cognisant of how even minor adjustments to recommender systems can expose users to further harm, these measures are restricted to services already carrying out on-platform tests who are medium to high-risk for at least two kinds of specified illegal harms (Ofcom, 2024). It is unclear whether such a measure provides sufficient cover, especially when services only have to refer to results from the safety metrics in future adjustments, which provides minimal incentive to act (Integrity Institute, 2024). Crucially, unlike analogous European legislation (discussed below), it doesn’t create incentives for platforms to engage independent auditors. The measure proposed by Ofcom is overly cautious, prioritising costs to services (Molly Rose Foundation), and reflects the limited consideration given by the regulator to mitigating the impact of recommender systems.
  5. Platforms in Category 1, likely including major social media services, face additional duties, but these focus more on empowering users to protect themselves from harmful content, rather than preventing harm from reaching vulnerable individuals or inciting violence in the real world. Category 1 duties include requiring platforms to offer user empowerment tools (Section 15, OSA). These tools allow adult users to opt-out of being exposed to content that isn’t illegal, such as abusive speech targeting characteristics like race or religion or inciting hatred based on such characteristics (Section 16, OSA). This is unlikely to be effective. Those adults most at risk from this content are the least likely to use the tools to filter out that content. Someone who has racist, antisemitic, homophobic or misogynist views is unlikely to choose to filter out that content. Therefore, they will see this content which will act as an echo chamber strengthening these views. People who don’t have these views are more likely to object when they see it and so use the tools to filter it out, giving them a false impression of a world where these views are less prevalent. In addition, the user empowerment tools are designed to protect the users of the service, not those who might be targeted by abusive content and affected by it in the real world (such as the victims in the Southport riots) but don’t have platform accounts.
  6. Category 1 platforms are also subject to additional transparency reporting obligations, which Ofcom could leverage to require greater transparency regarding the role of recommendation algorithms in the amplification of harmful content. The OSA grants Ofcom powers to compel certain services to be more transparent about their trust and safety measures. For categorised services, Ofcom must issue an annual transparency notice requiring providers to produce a transparency report. For user-to-user categorised services (including social media), this can include the “design and operation of algorithms which affect the display, promotion, restriction or recommendation of illegal content, content that is harmful to children, relevant content or content to which section 15(2) applies” (Schedule 8, OSA). This provision has the benefit of extending the transparency duties beyond illegal content to encompass the content covered by the user empowerment duties, including content that incites hatred against specific groups but may not constitute hate speech (as discussed above). However, the register of categorised services is yet to be set out in secondary legislation (it was introduced to Parliament on 16 December 2024), and the relevant transparency obligations are not expected to be in force until at least 2026. In its guidance for Category 1 services, Ofcom should clarify that, given the nature of recommendation algorithms, transparency notices could require third-party audits of these algorithms, with further clarification and guidance from the regulator on the process and independence of these audits.
  7. Crucially, for transparency obligations to effectively enable scrutiny of recommendation algorithms that can amplify harm, Category 1 services should include even smaller providers that deploy such algorithms. While Ofcom’s advice on Category 1 thresholds acknowledges that “content recommender systems” are a key factor in the rapid dissemination of content, its advice to the Secretary of State recommends limiting Category 1 to services with at least 7 million users (approximately 10% of the UK population) (Ofcom, 2024). This is concerning because smaller, fringe platforms—such Rumble, Odysee and Bitchute as seen in the Southport riots (ISD, 2024)—, can significantly amplify harmful content yet may fall outside Category 1 and thus be exempt from additional obligations, including the transparency reporting. Although the Draft Statement of Strategic Priorities (SSP) for online safety, published by DSIT in November 2024, urges Ofcom to “keep abreast of new and emerging small but risky services, which are posing harm to users online” (DSIT, 2024), the categorisation thresholds proposed by the Secretary of State in December 2024 contradict this goal by adopting Ofcom’s size-based advice and leaving smaller but risky services out of scope (Statement UIN HCWS312, 2024).
  8. The UK’s approach in the OSA can be contrasted with the one adopted by the EU Digital Services Act (DSA), which offers more comprehensive protection against large systemic issues. Unlike the OSA, the DSA adopts a broader approach, addressing “systemic risks” that extend beyond illegal content, including risks related to “any actual or foreseeable negative effects on civic discourse and electoral processes, and public security” (Article 34, para. 1, c, DSA). The Act requires the Very Large Online Platforms (VLOP) and Very Large Online Search Engines (VLOSE) to identify and manage these risks, as well as to hire outside auditors to conduct independent risk-assessments. Both company-led risk assessments and outside audits are mandatory and made public—documents that have the potential to provide relevant insights into how platforms operate, and the risks associated with their business models. However, this approach is not without its limitations. The first batch of documents published by the European Commission raises concerns about the adequacy and quality of these checks. Experts pointed out that auditors often relied on firms’ internal processes, raising questions about data collection, potential pushback, and the extent to which they examined beyond company-led compliance (Scott, 2024). The specific level of transparency was inevitably reliant on the goodwill of both companies and auditors, highlighting the need for additional guidance on the role of assurance processes if the goal is truly to understand the functioning of recommendation systems, the “magic sauce” companies are so protective of.
  9. The DSA includes provisions for platforms to quickly implement response measures in times of crisis, beyond their regular risk mitigation efforts. In “extraordinary circumstances” posing a serious threat to public security or health, the DSA mandates a crisis response mechanism for VLOPs and VLOSEs. These crises could stem from armed conflicts, terrorism, natural disasters, or pandemics. Triggered by a recommendation from the regulator, the European Board for Digital Services, platforms may need to take urgent measures, including adapting “relevant algorithmic systems and advertising systems” and “taking awareness-raising measures and promoting trusted information and adapting the design of their online interfaces“ (Recital 91). While it is unclear if the rapid spread of rumours and falsehoods alone would activate this crisis response mechanism, the DSA also introduces voluntary crisis protocols. These protocols encourage the largest platforms to develop specific response plans for situations requiring a rapid, cross-border response—which include situations when “platforms are misused for the rapid spread of illegal content or disinformation or where the need arises for rapid dissemination of reliable information” (Recital 108).
  10. In addition, the DSA explicitly mentions several obligations relating to recommendation algorithms for very large platforms and search engines, providing greater transparency and opportunity to assess risk associated with these systems. VLOPs and VLOSEs must consider whether and how the design of their recommender systems influences systematic risks (Article 34, para. 1, DSA). Particular attention must be given not only to illegal content but to the use of their services to disseminate or amplify misleading or deceptive content, such as disinformation (Recital 84, DSA). VLOPs and VLOSEs must mitigate these risks using “reasonable, proportionate and effective” measures that include the testing and adapting of recommender systems. While they are expected to be diligent in the measures they take, providers may need to mitigate the “negative effects of personalised recommendations” and correct the criteria used in recommendations (Recital 88). The DSA also requires VLOPs and VLOSEs to explain “the design, logic, functioning and testing” of their recommender systems to the relevant regulator—the Digital Services Coordinators or the Commission (Article 40, DSA). The usefulness of this provision in monitoring and assessing compliance is evidenced in the Commission already making requests to Tik Tok, YouTube and Snapchat (European Commission, 2024).
  11. Finally, the EU DSA introduces important provisions to foster research on the operation of recommender systems and to empower users. VLOPS and VLOSEs must provide access to data to vetted researchers to conduct research on the detection, identification and understanding of systemic risks and to assess the “the adequacy, efficiency and impacts of the risk mitigation measures” (Article 40, DSA). This adds an additional level of scrutiny that is essential in developing responses recommender systems. Additionally, the DSA ensures greater transparency for users into how platforms’ recommender systems operate by requiring all providers using such systems to explain in their terms and conditions why certain information is suggested (Article 27, DSA). Although the effectiveness of these provisions can be questioned, as users often skim terms and conditions, these requirements nonetheless empower users by mandating that platforms provide options to modify or influence how information is presented. Where multiple options exist, a functionality must be introduced to allow users to “select and to modify at any time their preferred option” (Article 27, DSA). To date, the DSA represents the strongest attempt to promote understanding of the functioning of, and address harms stemming from, recommendation algorithms.
  12. Overall, despite limitations in the law, the Online Safety Act regime could be adapted to more effectively address the risks associated with recommendation algorithms. While the OSA’s risk assessment and safety duties primarily focus on illegal and harmful content, other provisions could support Ofcom’s efforts to improve the understanding and mitigation of risks posed by recommendation algorithms. This includes leveraging transparency reporting powers for Category 1 services, and expanding the Codes of Practice to incentivise platforms to adopt systemic measures, such as crisis protocols, that go beyond analysis of individual pieces of content. Encouragingly, DSIT’s Secretary of State has acknowledged the role of platforms in the Southport riots, emphasising the “rampant spread of misleading information and incitement to violent disorder” and called for means to “hold platforms to account for their part in securing the UK online information environment and safeguarding the UK from future crises” (DSIT, 2024). Ofcom has also indicated it is working towards a consultation (to be held in Spring 2025) on expanding the Codes to include crisis response protocols for emergency events (Ofcom, 2024). In addition, the regulator is currently recruiting an Advisory Committee on Disinformation and Misinformation – which will hopefully examine issues beyond the narrow false communications offence in the act.

Expanding accountability for recommendation algorithms

  1. AI regulation could fill some of the gaps in online safety regulation regarding the specific risks arising from algorithms amplifying harmful content, but these risks are currently overlooked in the UK government’s proposed approach to AI. While the government has acknowledged the new and amplified risks of AI in disseminating false, biased, or harmful content—particularly focusing on AI-generated content like deepfakes (DSIT, 2024)—less attention has been paid to the risks of recommendation algorithms amplifying harmful content, which (as discussed above) can lead to harm to individuals and communities. Academics also often lack access to the necessary data to understand these risks. Research shows that scholars have been unable to access social media ranking algorithms, and the most extensive studies in the field are often conducted by the platforms themselves, raising questions about their independence and trustworthiness (Söderlund et al, 2024).
  2. These issues have been overlooked both in identifying risks and in developing effective policy and regulatory proposals. The UK government’s approach, from February 2024, which relies on sector oversight and voluntary commitments from large AI companies to pre-test frontier models, primarily focuses on future, existential risks. This approach is insufficient to address the current, real risks posed by algorithms already deployed by tech companies, including social media, video sharing, and search platforms. Current supervision powers have been focused on cross-sectoral risks and the autonomous risks of highly capable AI systems—with, for example, the AI Safety Institute focusing on the “understanding of the safety of advanced AI systems” (AISI, 2024). In other areas, the UK approach has been sector-based, more focused on empowering existing regulators to respond. While this has the benefit of being context-specific, enabling the adoption of more bespoke measures, it does require the appropriate sector regulator to understand the nature of the risks and develop appropriate responses. In addition, while this “pro-innovation” approach is considered more industry-friendly, as it relies on soft law, it is less effective in requiring companies to proactively identify and mitigate the risks posed by their systems.
  3. The UK’s soft law approach to AI regulation contrasts with the EU’s, where the AI Act requires providers to manage risks arising from AI systems, even though recommendation algorithms are not explicitly within its scope. The final version of the EU AI Act does not explicitly include platform recommender systems within the scope of high-risk AI systems—an amendment proposing this inclusion was not adopted (European Parliament, 2023, Amendment 740). However, the AI Act makes direct reference to the DSA’s systemic risks and mitigation efforts, stating that VLOPs and VLOSEs using AI recommender system face strict requirements on transparency, impact assessments, monitoring, reporting, and mitigating measures. They are required to assess potential systemic risks stemming from the design, functioning, and use and potential misuses of their services, and to take appropriate risk mitigating measures (Recital 118, EU AI Act). Therefore, in the EU, excluding recommendation algorithms from the AI regulation was possible precisely because extensive risk assessment and mitigation duties already operate within the DSA—the specific regulation focused on platform regulation. The UK lacks comparable duties.
  4. In the UK, it is crucial that Ofcom, as the sector regulator for online safety, has the powers and capacity to require companies to provide access to their recommendation algorithms, enabling the type of safety testing the AI Safety Institute currently conducts for AI foundational models. While Ofcom has made it clear that the Online Safety Act duties apply to some AI providers, the focus has been on generative AI chatbot tools and platforms, but only where it allows users to share content generated via chatbot with other users (being classed as a “user-to-user service”) or where it enables the search of more than one website and/or database (being classed as a “search service”) (Ofcom, 2024). Ofcom’s focus clearly remains on content—AI tools that generate potentially illegal or harmful content for children are subject to the Act’s duties. Unfortunately, this content-focused approach misses the mark when it comes to AI systems used by regulated services to curate, organise, and amplify content on their platforms. These recommendation algorithms have only briefly and insufficiently been mentioned by Ofcom’s interpretation documents (as discussed above).
  5. Future regulation focusing on AI systems could fill the gaps in the online safety regime. There are already legislative proposals to introduce statutory obligations. For example, Baroness Ritchie of Downpatrick, with the support of civil society organisations, has proposed an amendment to the Product Regulation and Metrology Bill to place risk assessment and mitigation obligations on providers of products containing AI systems—which could be applicable to social media platforms deploying recommendation algorithms. If passed into law, it would require the provider of a product or digital product containing or relying on an AI system to carry out a specific risk assessment relating to the impact of the AI system on the product or digital product’s functioning and use, and then take reasonable steps to reduce, mitigate, or manage the relevant risks. This would be nicely complemented by AI assurance from third-parties, which can help provide users, regulators and policymakers with the basis for understanding these systems. Indeed, it is promising that the UK government is committed to developing a “roadmap to trusted third-party AI assurance”, recognising the crucial role of assurance in enabling “industry to innovate at pace and manage risk” (DSIT, 2024). However, rather than being made on a voluntary basis, this should be put in a statutory footing and subject to the oversight of a regulator.
  6. Even with increased transparency of recommendation algorithms, legal and technical safeguards are crucial to enable audits by both regulators and independent researchers. Understanding and mitigating the risks of these algorithms may require disclosing training data, capability testing methodologies, and risk assessments. This presents a challenge, as the UK government has committed to protecting source code as intellectual property and agreed to no mandatory source code disclosure provisions in recent international trade deals, including those with Japan, the EU, Singapore, Australia, and the CPTPP (Jones, Kira & Tavengerwei, 2024). However, as Longpre et al (2024) argue, independent algorithmic risk auditing is essential for accountability, noting that “[e]valuating the risks of models that are already deployed and have millions of users is essential as the models pose immediate risks”. Therefore, we strongly recommend balancing the commitment to source code protection against the need for algorithmic transparency. This requires explicit carve-outs, not only in trade agreements but also in broader legislation, to grant public interest, independent researchers the necessary access to audit algorithms—including social media’s recommendation algorithms—for safety.
  7. Alongside increased access to algorithms for oversight entities, further research is needed to understand the necessary coordination across different levels of AI governance. This spans from overarching regulatory frameworks and granular standards for AI and social media companies to specific risk assessment and mitigation procedures for service providers. Research exploring how a duty of care framework can effectively address harms experienced by diverse individuals and groups while safeguarding democratic freedoms, and how external or third-party auditing processes can better align regulatory goals with the behaviour of AI developers and deployers, can fill crucial knowledge gaps.

 

III.      Recommendations

  1. Ofcom should include a “break-the-glass” mechanism in its codes of practice. This would require services within the scope of the Online Safety Act to respond swiftly to unfolding events by taking urgent measures, such as adjusting their recommendation algorithms. During moments of heightened social sensitivity, when emotional content likely to elicit engagement—and therefore be amplified by recommendation algorithms—is particularly prevalent within a community (as in the case of the Southport riots), social media companies might need to act quickly and adapt their content moderation responses. Ofcom should amend its illegal content code of practice to incentivise in-scope services that employ recommendation algorithms to develop a “crisis protocol” that could be implemented to coordinate responses to certain forms of violent extremist acts linked to online developments. This is particularly useful for encouraging cross-platform collaboration, as research highlights the role of cross-platform ecosystems in the rapid spread of harmful content during the Southport riots (ISD, 2024). While Ofcom’s guidance focuses on the illegality of content, the Online Safety Act provides a legal basis for the regulator to include more systemic response measures in its codes of practice. These measures should incentivise platforms to focus on their systems and processes—including, crucially, the algorithms they employ—and therefore could also help mitigate the spread of lawful content that nonetheless could cause harm. Ofcom has announced plans to consult on its code of practice in spring 2024, considering proposals for crisis response protocols for emergency events (Ofcom, 2024). This is a necessary step, but it should include measures to adapt recommendation systems.
  2. Ofcom should leverage its investigatory powers and the transparency notices for Category 1 services to require companies to grant the regulator technical access to their recommendation algorithms for auditing, including access via an API. A lack of transparency is the main obstacle to understanding and mitigating the risks posed by recommendation algorithms, including the rapid spread of misinformation and disinformation, which can have harmful real-world consequences, as seen in the Southport riots. Regulatory scrutiny of these algorithms is essential. Ofcom’s transparency authority under the OSA allows it to require companies to report on the “design and operation of algorithms which affect the display, promotion, restriction or recommendation” of content (Schedule 8). Crucially, Category 1 platforms should emphasise risky functionalities, not size, so that smaller user-to-user services that adopt high-risk recommendation algorithms should also be in scope of the additional obligations.
  3. The government should introduce legislation imposing legal duties on social media platforms to assess and mitigate the safety risks posed by their recommendation algorithms. AI regulation should address not only future or “existential” risks but also the impact of currently deployed algorithms, such as recommendation systems, which should be subject to mandatory safety scrutiny. This could be achieved through amendments to the Online Safety Act or new legislation, such as Baroness Ritchie of Downpatrick’s proposed amendment to the Product Regulation and Metrology Bill. These risk assessments should focus on algorithmic functionalities, rather than specific content categories. Crucially, they should consider the potential impact on different user demographics (for example, immigrants and asylum seekers) and broader societal implications, including on groups and individuals who are not platform users. Based on these assessments, platforms should be required to implement appropriate mitigation measures, including specific measures to be incorporated into crisis protocols. The effectiveness of these measures should be regularly evaluated and adjusted as needed. To increase accountability, companies deploying recommendation algorithms should be transparent about their risk assessment processes, their findings, and the mitigation measures implemented.
  4. Future AI regulation should require that recommendation algorithms deployed by social media companies undergo scrutiny by external auditors. Independent evaluation of these algorithms is crucial for understanding their functioning and vulnerabilities, and for providing accountability—with independent audits being widely regarded as essential for ensuring safety, security, and trust (Algorithmic Justice League, 2022). For audits conducted by third parties on behalf of platforms, rules structuring and guiding assurance processes are needed to ensure they are adequate and effective capable of holding platforms to account. Independent audits by researchers should be conducted in the public interest and adhere to responsible research and innovation (RRI) principles. In the UK, Ofcom could collaborate with the AI Safety Institute, leveraging its expertise in testing AI model safety and established trust with AI companies, to audit social media algorithms. More research is needed on the role of independent external audits to understand their limitations and potential, and explore how insights into assurance processes within social media and AI regulation can inform effective regulatory frameworks and enhance accountability.
  5. Platforms should carve out exceptions in their policies and lawmakers and policymakers should include safe harbour provisions in legislation to allow researchers to conduct safety assessments of recommendation algorithms. While access restrictions and protections adopted by AI and social media platforms are primarily aimed at malicious actors, they have the unintended consequence of restricting public interest research and shielding systems from scrutiny (Longpre et al, 2024). Legal and technical protections for public interest research would empower independent researchers to audit recommendation algorithms and assess a range of risks without facing legal repercussions for their investigative activities. Safe harbours should be granted both through carve-outs in platform policies—as part of their commitments to improve product safety—and by lawmakers and policymakers, including through inclusion in future AI regulation. This may also require reforms to intellectual property laws, trade agreements, and computer misuse and fraud legislation.

 

18 December 2024

 

IV.      Authors

This submission of evidence was prepared by the following law and computer science scholars from the Sussex Centre for Law and Technology (SCLT) and SussexAI at the University of Sussex. These experts are available to provide further information or oral evidence at a Committee hearing, if required.

 

About the University of Sussex’s Centre for Law and Technology (SCLT)

An international hub for research, teaching, and engagement in law and technology, the University of Sussex’s Centre for Law and Technology houses leading scholars with expertise in technology and information regulation, global governance of technology, intellectual property, and legal innovation. We conduct cutting-edge research through collaborative projects with policymakers, civil society organisations, and industry leaders.

 

About SussexAI – Centre of Excellence in Artificial Intelligence at the University of Sussex

Sussex AI is an interdisciplinary Centre of Excellence at the University of Sussex. Our centre draws together world-leading experts to create a critical mass of technical skills and domain knowledge and proving that a challenge is only impossible until it’s done. Our research and training covers a wide range of topics in AI and data science, with a unique multidisciplinary Sussex angle.