Written evidence submitted by the UK Safer Internet Centre (UKSIC) (SMH0044)
The UK Safer Internet Centre (UKSIC), established in 2011, is a leading global partnership helping to make the internet a great and safe place for everyone. We provide support and services to children and young people, adults facing online harms, and professionals working with children.
A bridge between Government, industry, law enforcement and society, we are the engine of the online protection landscape in the UK, dealing with both prevention and response.
Formed of three charities, Childnet, Internet Watch Foundation and SWGfL, we work together to identify threats and harms online and then create and deliver critical advice, resources, education and interventions that help keep children and young people, and adults, safe. We share our best practices across the UK and globally.
We focus our work around four functions:
Where we provide advice and support to children and young people, parents and carers, schools, and the children’s workforce.
Which provide support to professionals working with children and young people with online safety issues, and support to all adults facing issues with harmful content and non-consensual intimate imagery online.
Which provides an anonymous and safe place to report and remove online child sexual abuse images and videos wherever they are found in the world.
We operate a Youth Advisory Board, and we nurture youth participation, providing a focus on youth voice to give young people agency to make a difference in their school communities.
UKSIC is the proud coordinator of Safer Internet Day in the UK.
Our partners the IWF and the SWGfL have submitted responses to this consultation and we fully endorse their responses. We have summarised and integrated their responses where appropriate. Please refer to their full responses for the detail.
Summary:
The UK Safer Internet Centre (UKSIC), is submitting evidence to the committee hearing on “Social media, misinformation and harmful algorithms.” Please find below the summary of our key arguments and the full response to the questions further below.
Algorithms on social media platforms, especially recommender systems, significantly influence the spread of harmful content. Studies show that such systems promote misogyny, violence, and extremist ideologies, with alarming consequences, such as normalising harmful behaviours in schools and desensitising children to violence. Safety-by-design should be the priority in the implantation of any new functionalities or services to reduce the potential harm to the user.
The proliferation of harmful content online has detrimental effects on children and young people. Research from multiple sources reveals exposure to ultra-violent and sexually explicit content, with some platforms delivering such material within seconds of engagement. This exposure negatively impacts children’s emotional and psychological development, intensifies feelings of vulnerability, and increases the risk of offline violence.
Current reporting mechanisms on platforms are underutilised and inaccessible, particularly for children. The lack of effective tools for reporting harmful content and addressing complaints leaves users unsupported. UKSIC highlighted the effectiveness of its helplines, such as the Professionals Online Safety Helpline (POSH), in directly engaging with platforms to achieve faster and more consistent removal of harmful content.
The upcoming Online Safety regime, including the Online Safety Act, provides an opportunity to implement safety-by-design principles. UKSIC calls for the inclusion of violent content as “Primary Priority Content” to ensure its removal from recommender systems. The framework should focus on enhanced reporting mechanisms, including the use of trusted flagger systems, and establish clear accountability for platforms.
The response also underlined the role of NGOs like UKSIC in informing policy through insights gathered from helplines, ensuring that regulatory measures are grounded in real-world user experiences. This is crucial to combat misinformation, harmful algorithms, and the rapid evolution of digital threats effectively.
The submission drew attention to recent incidents, such as the Summer Riots, which were fuelled by online misinformation. These events underscore the need for swift implementation of protective measures to prevent similar harm in the future.
Call to Action for Policymakers
UKSIC urges policymakers to act decisively in implementing and refining the Online Safety Act, with an emphasis on protecting young people and fostering a safer digital environment. Parliamentarians are encouraged to support enhanced reporting mechanisms, greater transparency from platforms, and the adoption of safety-by-design principles to address the growing challenges posed by misinformation and harmful online content.
Questions:
To what extent do the business models of social media companies, search engines and others encourage the spread of harmful content, and contribute to wider social harms?
The Online Safety Act offers us a unique opportunity and positioning to not only change the existing harmful functionalities but also design choices. In the current profit-driven business strategy that is mainstream in the tech industry, we are very pleased to see that the new government update and in particular Ofcom and DSIT are making Safety-by-Design a key outcome to come out of the implementation of the OSA.
As UKSIC we are concerned with the online content that is easily accessible by children on social media. We would also like to acknowledge the inclusion of recommender systems alongside advertising business models as a function that can greatly exacerbate the risk of accessing online harmful content. Children are affected by the content they consume online, so measures need to be taken to ensure that their experience online is safe and age appropriate.
In recent years, particularly with the introduction of short video form content, the effect of function systems plays a significant role in children. In a recent study published by UCL7 there was clear evidence that hateful ideologies and misogynistic tropes that were shared online and massively spread with the help of the algorithm, have moved off screens and into schools, becoming embedded in mainstream youth cultures. Vodafone also conducted research which showcased significance evidence that AI recommender systems are probing young people into harmful and extremist content; “on average, boys aged 11-14 are exposed to harmful content within 30 minutes of being online and one-in-10 are seeing it in as little as 60 seconds. This worrying trend stems from AI algorithms pushing content promoting misogyny (69%) or violence (79%) to boys following innocent and unrelated searches (59%)”. The Safer Internet Day research9 provides also insight into the understanding of children online of recommender systems: 71% of children that participated in the SID 2024 research told us: “we understand that when they ‘like’ or watch something online, it influences what content is suggested to them in future. 62% understand that algorithms choose the content they see in their feed or games and videos that are recommended to them. Particularly worrying also for extreme pornography which is easily accessible by children and often present in social media platforms which are largely used by children.
Research conducted by Crest Advisory in 2022 explored the relationship between social media and youth violence. It revealed growing concerns among parents, young people, and professionals like teachers and social workers about how online exposure influences offline behaviour.
Further research, undertaken in collaboration with the Youth Endowment Fund (YEF) and published in November, delved deeper into the issue. A national survey of over 2,000 young people across England and Wales examined their experiences with violence and vulnerability. Within a year, 55% had witnessed violence on social media, and 65% reported altering their behaviour to protect themselves from violence.
While general exposure to violent content did not significantly affect young people’s perceptions of violence or their sense of personal safety, exposure to sexually violent content had a pronounced impact. Young people exposed to such content were more likely to believe violent crime was on the rise and to skip school due to feeling unsafe. The YEF survey further underscored the complex relationship young people have with social media. While 79% of respondents believed social media contributed to youth violence, 23% of those who had experienced or witnessed violence offline reported increasing their time online out of fear of violence in their communities.
This dual role of social media—as both a perceived refuge and a source of risk—presents a significant challenge for policymakers and practitioners. For some young people, online content may provide little impact on their sense of safety; for others, it exacerbates feelings of vulnerability and fear.
Additionally, the latest research by the YEF and the Guardian illustrates that One in four teenagers who see real-life violence, including fist fights, stabbings and gang clashes, online are being served the clips automatically by algorithmic recommendation features with TikTok being the most likely place for teenagers to encounter real-life violent content, followed by X, according to the survey of more than 10,000 13- to 17-year-olds.
It is therefore of great importance to ensure that current social media platforms particularly large ones, provide a safety-by-design framework for the operation of their recommend systems which are safe for children, while at the same time providing the technological foundation for smaller organisations to use to ensure that in turn their recommend systems do not harm children.
How do social media companies and search engines use algorithms to rank content, how does this reflect their business models, and how does it play into the spread of misinformation, disinformation and harmful content?
Particularly alarming issue is the ease of access to harmful content on social media platforms popular with children. This poses significant risks related to several priority offences, including threats, abuse, harassment, and the potential escalation to terrorism offences. Recommender systems can indoctrinate children into extremist views by continuously pushing such content.
Indeed, reports such as those by Ribeiro et al. (2020), and Amnesty International (2023) have highlighted how the affordances of social media platforms, such as YouTube's recommender systems, actively amplify and direct harmful content. This includes content associated with the alt-right, misogynist and manosphere content (Reset Australia, 2022), and self-harm material (Amnesty International, 2023). In a worldwide first, a recent ruling by a UK coroner in October 2022 identified Instagram as ‘likely contributing’ to a young person’s death due to the high level of self-harm material recommended on her feed in the months before she died.
Online echo chambers and the rapid rise and circulation of electoral fake news, mis/disinformation, and polarisation towards political extremism on the internet have had particular impacts on young people (Lewis-Kraus, 2022). Most recently, misogynist influencers, who have leveraged their fame to promote polarised far-right extremism, have utilised the phenomenon of echo chambers to bring gendered hate to prominence in other dimensions of public discourse, such as boys’ misogynist behaviours within schools and educational settings (Das, 2022).
Social media has introduced a profound and currently unmanageable challenge in relation to violence, amplifying its prevalence and impact. Organised gangs exploit these platforms to recruit young people into illegal drug markets and violent activities, provoke rival groups, desensitise users to acts of violence, and foster a culture of callousness and hostility. Social media also plays a pivotal role in escalating conflicts and coordinating violent encounters. This dynamic has reshaped youth culture and lifestyles in deeply harmful ways.
Through these platforms, violence is often celebrated, glamorised, and normalised, further perpetuating real-world harm. Addressing this issue requires a robust and comprehensive state response. This should include enhanced monitoring of social media activity, swift removal of harmful content, and stringent consequences for those who create or disseminate such material.
What role did social media algorithms play in the riots that took place in the UK in summer 2024?
We are eagerly awaiting to see the Ofcom Children’s Harms Update which is expected within the first months and we sincerely hope that the provisions for Recommender Systems have been updated to also include violent content. More specifically, RS1 measure on the Illegal Harms Codes, which aims to “Ensure that content likely to be Primary Priority Content is not recommended to children” we propose to prevent Primary Priority Content to being recommended to any user, completely removing csam, intimate image abuse and terrorist content from platforms. With the current provision and age assurance technologies which have not yet been deployed, children may still easily falsify their age, allowing them to access inappropriate and harmful content. RS2 also refers to the reduction of priority content rather than the removal of such content within the recommender feed system. Children will still be able to view content that is harmful, and could have intense psychological impact on them including, suicide content, body dysmorphia, and online challenges which have cost the lives of so many children in the UK in the past.
We would like to see violent content to be included in the Primary Priority Content. RS2 is particularly worrying, as children will still be able to view violent content within recommender systems since the provisions aims to reduce the content rather than block it or remove it. The current trends are worrying, and significant provisions should be introduced the minimise the sharing of violent and harmful content. According to the POSH 2023 report, during March 2023, there was a lot of attention and a volume of cases around “TikTok riots” in schools², which reflects concerns across the education sector about the use of social media platforms to host content related to behaviour in schools. These incidents coincide with a number of cases specifically referring to these incidents, with enquirers requesting help to get content taken down. Regarding the location of content that caused concern (and was often requested to be removed), it is no surprise, that the overwhelming majority was hosted on social media platforms. We therefore would like the reinforce the need to blocking and removing access to violent content which can have significant psychological and mental effects on children who come in contact.
How effective is the UK's regulatory and legislative framework on tackling these issues?
Reporting Mechanisms are an important part of the Online Safety Act, and the implementation of the service requirements. The current provisions with regard to reporting mechanisms exist in most large service providers, however, they are often quite underutilized and are not accessible for children. For instance, in the Online Nation 2024 Ofcom report, “Ofcom’s VSP tracker in August 2024 found that 51% of VSP users aged 13+ said they had used a flagging and reporting mechanism in the past, an increase from 42% in February 2024. Of those who had used reporting and flagging mechanisms, 89% said they would use one again, consistent with February 2024 (87%). The main reasons users said they would use them again were that they were easy to use (61%) or easy to find (50%); again, consistent with February 2024 (66% and 56%).240 Forty-seven per cent of parents, who were aware of flagging and reporting mechanisms, said these should be made easier for children to use and access, while 9% said they were already easy enough for children to use and access, similar to February 2024 (50% and 8%)”.
ADR and Reporting Mechanisms
The lack of a structured approach to alternative dispute resolution (ADR) in the proposals represents a missed opportunity to bolster user trust and platform accountability significantly. ADR offers numerous advantages, such as easing the burden on formal complaints processes, fostering more positive relationships between platforms and users, and potentially resolving conflicts in a manner that respects the interests of all involved parties. Additionally, ADR mechanisms like mediation, arbitration, or ombudsman services can bring expertise and impartiality that may not always be present in platform-driven complaints procedures. SWGfL suggests that the proposals could be enhanced by explicitly integrating ADR mechanisms into platforms' strategies for addressing complaints and disputes. An outline of an ADR solution previously proposed by SWGfL can be found in volume 3 above. This could be supplemented by the development of specific guidance or standards for ADR mechanisms within the context of online harms. This would include criteria for mediators or arbitrators and processes that ensure fairness, transparency, and accessibility. Referencing Report Harmful Content, the Draft Online Safety Bill (Joint Committee), in December 2021 recommended (paragraph 457) that; “The role of the Online Safety Ombudsman should be created to consider complaints about actions by higher risk service providers where either moderation or failure to address risks leads to significant, demonstrable harm (including to freedom of expression) and recourse to other routes of redress have not resulted in a resolution” and that “We suggest that the Department look to Report Harmful Content as a potential model for what such an Ombudsman could look like”. While the proposals in Chapter “Reporting and complaints” establish a framework for reporting and complaints, the integration of ADR mechanisms could significantly improve the effectiveness, accessibility, and user trust in these processes. Leveraging SWGfL's expertise in online safety underscores the pivotal role ADR can play in the broader ecosystem of reducing online harm and resolving disputes.
Children have also displayed a significant lack of trust in existing reporting mechanisms. Many avoid flagging harmful content, fearing that doing so will lead to more of the same being recommended by algorithms. Others report frustrations with complex reporting processes, inconsistent platform approaches, and generic responses that fail to address their concerns. As we reiterated in our Protecting children from harms online” Blocking should become a required safety provision that should be included in services where children are present. Childnet’s research in 2021 showcased that “Many young people find blocking is a useful tool in response to being worried or upset about something online – they are more than twice as likely to block someone online (44%) as report them (21%). Only 17% of 11-year-olds said they would report”. This clearly showcases the importance of ensuring that blocking remains an option for children in all platforms including small services”. Blocking is a more effective tool, and the omission of it in the recommended functions, provides a” fertile” and dangerous ground of grooming, harassment, cyberflashing which could all harm children significantly.
Children should not have to accept violent content as an inevitable aspect of their online lives. Safer Internet Day research sends a clear message to tech companies now is the time to act. By implementing robust safety measures and fostering age-appropriate online environments, platforms can ensure that children’s digital experiences are safer, healthier, and more constructive.
Children report that encountering violent content online has become "unavoidable," with exposure beginning as early as primary school, often before they meet the minimum age requirements for popular platforms. New research commissioned by Ofcom reveals that all participating children had come across violent material online, frequently via social media, video-sharing sites, or messaging apps. This content includes violent gaming, verbal discrimination, and footage of school or street fights, the latter often normalised among peers as a means to build social status or avoid being labelled "different."
While more extreme content—such as gang-related violence—is encountered less frequently, its existence highlights the broader spectrum of harmful material available online. Alarmingly, children feel powerless to control their exposure to such material, often citing group chats, strangers' posts, or algorithm-driven recommendations as key contributors.
The study underscores the role of peer pressure, particularly among teenage boys, who are the most likely to share and seek out violent content. Sharing such material is often seen as a way to "fit in" or gain online popularity due to the high engagement it generates. Younger children, aged 10–14, describe feeling pressured to not only watch violent content but to respond positively to it, fearing social isolation if they don’t. By contrast, older children appear more desensitised and less likely to actively share violent material. Awareness campaigns should therefore be reinforced to target young boys and prevent the spread of a “violent culture” in younger boys.
Strengthening children’s media literacy is therefore a critical component of mitigating harm. Ofcom’s forthcoming three-year media literacy strategy will focus on equipping children with the skills to navigate online spaces safely and critically. A parallel report by Magenta highlights how highly media-literate children proactively reduce their exposure to harmful content, even in the absence of effective platform protections.
Which bodies should be held accountable for the spread of misinformation, disinformation and harmful content as a result of social media and search engines’ use of algorithms and AI?
The existing mechanisms and business practices that are put in place are not entirely sufficient to offset the volume of violent content that is produced and shared online. That means that a significant number of children have access to violent and harmful content on large and easily accessible services. The latest research, from the French Safer Internet Centre, highlighted a concerning trend among school-age children related to the exposure to ultra-violent scenes—such as torture, beheadings, and war—on their smartphones, with 25% of 11 to 18-year-olds engaging in this behavior, often as part of challenges on platforms like Reddit, Telegram, Discord, or X (formerly Twitter), where they test their limits by watching traumatic content, including footage of drug cartels, terrorist attacks, and brutal murders. The research also concludes that “Exposure to ultra-violent content can severely impact a child's emotional and psychological development, much like unprepared exposure to sexual or pornographic scenes. Such content can desensitise children to violence, leading to aggressive behaviours or emotional disturbances. While platforms claim to filter violent content, these measures are often insufficient.” The role of the regulator and notably Ofcom is to ensure that the Online Safety Act is effective, flexible to match the rapid changing nature of tech, and also achievable. The provisions proposed should hold service providers accountable, for their design and functionality choices which might harm the users. As noted by Minister Peter Kyle, specific emphasis should be placed on the “Safety-by-design" principle when designing features, applications and services.
The evolving use of social media platforms among young people, as highlighted by data such as OFCOM’s 2023 Media Literacy report, shows that while YouTube remains the most popular platform, over 50% of young people aged 3 to 17 are now using TikTok. This shift in platform usage underscores the need for robust systems of reporting and accountability, particularly as harmful content continues to proliferate online.
The Professionals Online Safety Helpline (POSH) has demonstrated the effectiveness of direct relationships with major platforms, enabling faster and more consistent takedowns of harmful content. For example, 68.8% of cases handled by the helpline in 2023 were successfully referred to platforms for review, often achieving better outcomes than individuals attempting to report content independently. Despite these efforts, existing mechanisms and business practices remain insufficient to address the sheer volume of violent and harmful content being shared.
In light of these challenges, the upcoming Online Safety regime provides a critical opportunity to establish clear avenues for reporting and accountability. This includes leveraging tools like the proposed trusted flagger system and assessment reports to ensure harmful content is swiftly identified and addressed. NGOs such as the UK Safer Internet Centre (UKSIC) play a pivotal role in this framework by collecting valuable insights into the actual experiences of users and the harms that occur online through their helplines. The Professionals Online Safety Helpline (POSH) and the Revenge Porn Helpline have demonstrated the capacity to facilitate the reporting of harmful content directly to platforms, yielding better outcomes than individuals reporting independently.
By integrating these insights into the broader online safety framework, policymakers can ensure that strategies are grounded in the lived experiences of users, particularly children and young people. Implementing comprehensive measures that prioritise their safety and emotional wellbeing will not only address existing gaps but also set a precedent for creating a safer and more supportive digital environment. Setting a regulatory framework to prevent the spread of misinformation online is a crucial part of the Online Safety Act and the online safety regime should be supported and enhanced to support the increased volume of information sharing through algorithmic systems. Notably, the Summer Riots are evidence on the need for reinforced protection and safety measures to prevent online harm. The recent violence across England, fuelled by far-right misinformation online, prompted London Mayor Sadiq Khan to call for urgent revisions to the Online Safety Act, stating that existing laws are “not fit for purpose.” He criticised the spread of fake news, including a notorious a fabricated article shared by X owner Elon Musk, which exacerbated tensions. The riots, incited by false claims about a Muslim immigrant, highlight the critical need for stronger frameworks to combat misinformation and protect communities from its harmful consequences.
18 December 2024