POSR0014

Written evidence The Royal College of Psychiatrists

1. About the Royal College of Psychiatrists

1.1              The Royal College of Psychiatrists is the professional medical body responsible for supporting psychiatrists throughout their careers from training through to retirement, and in setting and raising standards of psychiatry in the United Kingdom. We work to secure the best outcomes for people with mental illness, intellectual disabilities and developmental disorders by promoting excellent mental health services, supporting the prevention of mental illness, training outstanding psychiatrists, promoting quality and research, setting standards and being the voice of psychiatry.

2. Background

2.1              On January 2020 the College published our report titled “Technology use and the mental health of children and young people.” This report explores the current evidence around the effects of the use of technology among children and young people. As well as looking at the impact of screen time, the report also looks at the impact of different types of screen use on the mental health of children and young people including:

• Negative content.
• How vulnerable groups may be affected such as those with mental health Problems and very young children.
• Potential for bullying and safeguarding issues.
• The potential for addiction.

2.2              In the report we made recommendations, including a number aimed at government and technology companies to protect users from the risks associated with the use of technology. The report also provided practical guidance to children and young people, parents and carers, clinicians and teachers on this issue. When we launched the report we became one of the first groups to call for regulators to have power to impose fines of up to 10% of global annual turnover of social media companies.

3. Ofcom

3.1              The Online Safety Bill gives Ofcom a lot of new responsibilities and power, far more then was original envisioned for the regulator. The government must make sure that Ofcom receives investment proportionate to their task otherwise many of the positive protections introduced by the Bill will never be properly enacted.

3.2               Specifically we need to make sure there is investments in people with expertise and skills. However, as Ofcom expands in response to these new duties, we have concern there may be increasing fragmentation within Ofcom.

3.3               We are also concerned that Ofcom is being asked to regulate on areas which have not been clearly defined. For example, we are concerned that terms such as harm do not have a clear definition.

3.4               As this is the first time legislating in this space, there will need to be a lot of learning in this process. The Government needs to make sure they are holding regular reviews of the work of Ofcom to understand how it is regulating and understand the impact of the legislation.

3.5               Effective regulation necessitates collaboration and coordination with other regulators, including globally, to have the required impact. Existing groups, including the Advertising Standards Agency (ASA) and the Information Commissioner's Office (ICO), potentially have a large degree of overlap in responsibilities. We seek clarification on how these groups will work together, and what knowledge is being shared between different agencies.

3.6               We also believe that there is an opportunity for Ofcom to learn from existing legislation in other countries. Specifically the European Union’s Digital Services Act.

3.7               We hope Ofcom is given the capacity to look at more preventive measures to protect people from encountering harm in the first place, and that areas which are not currently covered by the Bill, in particular the impact of loot boxes in children and young people, are reviewed

4. How service design can factor into the effectiveness of online safety regulation

4.1               Services may not have in place adequate systems to identify and remove harmful content as soon as possible. Safety technologies may be costly for innovative start-ups, leading to the risk that only very large, profitable companies can be compliant. Systems for sharing safety tech could support innovation. Examples of design features that could support the better identification of harmful content include:

• Media literacy resources, including prebunking games e.g. Bad News, that help users understand what is harmful content, and its impact upon themselves and others.
• Terms of service that users can engage with, and help users understand what may be harmful behaviour, even if not illegal.
• Rewards/incentives for appropriate reporting, that empower users to report, and give them sufficient feedback on impact of reporting (possibly via transparency reports).
• Ensuring that reporting flows are easy for users and enable users to report with feedback, and give moderators enough detail to make effective decisions.
• Helping users understand the range of moderation outcomes, if they want to flag concerns but may not want a user banned.
• A stepped approach to moderation for legal but harmful content, with non-punitive language in messages to users, to help users who breach terms of service understand better, and are helped to find alternatives/change their behaviour. This is especially important regarding the posting of self-harm content.
• Systems for rapidly evaluating trending content of potentially risky, imitable content such as a risky hashtag challenge to reduce copycat behaviours.
• Expert training of Artificial intelligence/Machine Learning for identifying risky content. There is a need for these systems of detection to evolve.

4.2               Services can act as delivery mechanisms for content: textual, aural, or visual materials. The information conveyed by this content may create the risk of harm, and this risk may be exacerbated by the service design itself. For example:

• Amplification of trending content that the services spread, that could appear in the newsfeed of all users.

• Amplification of negative content impact to individual users via algorithmic design or automated processes and recommender systems (i.e. algorithms inappropriately promote potentially harmful content to users at risk).
• Amplification of negative content via lack of warnings on such content or inappropriate warnings on such content (i.e. service design fails to adequately warn users regarding harmful content; users are not sufficiently literate to understand the impact of hashtags). Flagging of risky content needs to be done in a way that it is easy for users to recognise what might be risky content.
• Amplification of negative content impact via inappropriate accessibility of such content to vulnerable groups (i.e. service design makes harmful content inappropriately available to at-risk users, possibly via inappropriate algorithmic design, including ad managers and segmentation tools).

4.3               There are a number of design features and operational systems that can leave users feeling helpless and disempowered. These include:

• Data profiling such as affinity profiling that may distort a user’s newsfeed/search results.
• Dark patterns of nudge and persuasion, diminishing choice.
• Absence of tools to influence (‘clean’) algorithms.
• Extreme but legal content that has a powerful emotional impact that disrupts user critical thinking, especially when from charismatic influencers (ideological persuasion). Flagging paid influencers (akin to paid Google ads) may improve media literacy regarding influencers (but needs testing, as it could increase user attempts to become an influencer).
• This is an area where there needs to be close collaboration with the ASA and clarity in each originations role.
• Insufficient feedback on any reporting, leaving the user feeling that it is pointless to report.

4.4               Linked to concerns regarding the nature of content delivered via services, there are concerns that services may be designed in a manner which leads to excessive engagement. Such overuse may also exacerbate any potential risks associated with the content itself. For example:

• App design features that may exacerbate overuse, leading to addiction-like symptomology (i.e. apps are designed in such a way that promotes a degree of engagement that is potentially harmful to users).

4.5               Design may also expose users to commercial exploitation­. In addition to the under-represented risks of scams and cybercrime in the Online Safety Bill, there are three main ways in which users may be exploited commercially:

1. the payment of influencers and e-sports stars incites vulnerable and young people to pursue achieving influencer status, effectively becoming free content generators, or if paid, labourers. This is especially important for young people, who are not protected by regulations (see performance regulations for child actors, DfE) to ensure good care and access to education. This also applies to parents who are influencers and involve their children
2. App and Game design features that may exacerbate overspending, leading to associated harms (e.g. in App purchases, loot boxes, gamblification of games, predatory monetisation, including content on cryptocurrencies). This is not currently covered in the Bill and needs to be closely monitored.
3. Complex systems that make it difficult to set-up monitoring of spend, insufficient systems of feedback on spend, cancel a linked credit card or pursue a refund.

5.     Data required to understand different types of harm

5.1               We lack the kind of data that we need to draw reliable dose-response relationships between the presence of specific design features and the creation of online harms. This lack of data will greatly impact the ability for online safety regulation.

5.2               We suspect that many design processes can amplify such harms, but without industry data sharing on a large scale it’s very tough to isolate and quantify the ‘active ingredients’ here in a reliable way. For example business models often depend on advertising revenue. Advertisers want to see engagement, and extreme content is engaging; so the creator of extreme content is paid for that activity, and the recommender systems amplify it, because it is engaging.

5.3               We welcome the commitment in the Bill for Ofcom to produce a report on the ‘extent persons carrying out independent research into online safety matters are currently able to obtain information from providers of regulated services, exploring the legal and other issues which currently constrain the sharing of information for such purposes, and assessing the extent to which greater access to information for such purposes might be achieved.’ However, while this is an important step this may take up to two years to produce. The government should prioritise this report and develop a protocol for data sharing and a relevant ethical framework for informed consent as soon as possible. Currently data is mined without explicit user consent, and no framework exists as a guidance for research ethics committees.

5.4               One issue is the opacity of the technology, of not understanding the how and why you get the content you do. Qualitive research, including ‘ Young people’s online engagement and mental health: the role of digital skills’ showed the impacts, particularly in relation to self-harm, eating disorders, sexual abuse and excessive use was as follows:

“young people with mental health needs are not vulnerable due to a lack of digital skills; they are highly skilled, but these skills are not sufficient due to:

• Platform processes that spread content
• Processes that centre on the individual, including:
• Data profiling.
• Algorithmically managed feeds.
• Recommendation systems.
• Dark patterns (forced choice) of nudge and persuasion.

The report concludes “risky-by-design complex and opaque technological systems overwhelm the young people: there’s much that they cannot manage…  For those with mental health difficulties, the consequences can be extreme.”

October 2023