17              HUMAN RIGHTS AND THE REGULATION OF AI – 5Rights EVIDENCE

SEPTEMBER 2025

WRITTEN EVIDENCE SUBMITTED BY 5RIGHTS FOUNDATION

(RAI0074)
 

Introduction

5Rights Foundation welcomes the Joint Committee on Human Rights’ inquiry into the creation and delivery of rights-respecting regulation for artificial intelligence (AI).

As early digital pioneers, children are already using AI technologies in all aspects of their lives – for socialising and connecting with others,[1] playing at home and learning at school,[2] or to simply try it out.[3] This early uptake, their natural curiosity and still-developing cognitive function means children are more vulnerable to risks of harm that AI technologies can pose.[4][5] Despite this, children are largely absent from conversations around both the risks and opportunities AI can have on their lives.

Since its inception, 5Rights has advocated for an online world which treats children the same way we do offline. This means recognising their rights,[6] in particular their right to safety, privacy, participation and right to life – ensuring the digital world is built with this in mind.[7] Practically, this means ensuring digital services and products, including AI, are safe and age-appropriate for them to use.

We develop new thinking, innovative frameworks and technical standards to challenge received narratives of tech exceptionalism with a focus on implementable change. In 2018-2019, we participated as a member of UNICEF’s Artificial Intelligence and Children’s Rights Working Group, which explored the potential impacts that AI is having, and may have, on children in the future.[8] In 2022, we published our Shedding Light on AI report,[9] which provides a four-step process for the oversight of AI systems.

Earlier this year we published our Children & AI Design Code,[10] which provides a framework for the design of AI systems with children’s rights and needs in mind: helping AI developers to identify, evaluate and mitigate the risks and vulnerabilities of children upstream by stipulating nine criteria for developers to consider throughout an AI product’s lifecycle.

Developed alongside computer scientists, academics, experts and young people themselves, the Code focuses on practical and implementable measures for developers, which, taken together, are compatible with existing legislation and regulatory frameworks globally.[11]

Further, 5Rights’ research centre with the London School of Economics, the Digital Futures for Children Centre,[12] is seeking to capture the views and experiences of children and AI[13] as – despite its use in almost every aspect of their daily lives – they are not represented in wider policy decisions about its integration into society.

Summary

• Children are early adopters of AI, with unique vulnerabilities and entitlements that require specific consideration across the lifecycle of an AI system.
• AI technologies pose serious risks of harm to children that undermine their rights. This includes the use of their data to profile them, undermining their right to privacy, unfair decisions reached by systems that are not fair and its deployment in certain contexts (e.g. in schools) which may undermine their right to education.
• Government policy on AI has largely overlooked the distinct opportunities and needs for children. This is despite the fact that, for children, AI is now ubiquitous within their daily lives.
• AI regulation must set common criteria for the design of AI systems and their governance. Within any regulation, outcomes which ensure that children are protected, their rights upheld, and their diverse needs accounted for must be at its core.
• In order to ensure children’s rights are upheld everywhere, and their needs accounted for, both the public sector and private sector must adhere to standards set out in any future AI legislation. This should include a set of criteria that requires that AI systems are made, among other things: developmentally appropriate, safe, fair, transparent and accountable.
• International treaties, such as the Council of Europe, and regulatory frameworks in other jurisdictions, including the EU, embed front and centre that children’s rights must be upheld in the development of AI systems. The UK, in implementing its obligations under the Convention and in developing its own regulations, must ensure that children’s rights are at its core, with robust mechanisms for the transparency and accountability of AI.

Human rights issues

1. How can AI affect individual human rights for good or ill, in particular in the areas of privacy and data usage; discrimination and bias; and effective remedies for violations of human rights?

Children’s rights are additional to human rights, providing an additional layer of protection next to general human rights.

The UN Convention on the Rights of the Child,[14] ratified by the UK in 1991 and one of the only treaties to receive near-universal ratification globally, and its General comment No. 25 (2021) on children’s rights in relation to the digital environment,[15] establishes that children have the right to be protected from risks of harm and exploitation,[16] the right to privacy[17] and respect for their development and evolving capacities.[18]

The reason children have their own rights set out in the UNCRC is to recognise children require distinct protections and needs, safeguarding their civil, political, economic, social and cultural rights.[19] This is not any less true in the digital world.

Children are uniquely vulnerable to the risks posed by AI technologies owing to their still-developing cognitive and reasoning function.[20] This means they require additional protections to keep them safe as they mature and develop their understanding of the world and critical thinking skills. Children also develop uniquely and at different rates, meaning that certain children will require specific protections at different stages of their development.[21]

In the digital world, the risks faced by children can be categorised through the 4Cs (or 5Cs) framework: content, contact, conduct and contracts (commercialisation), as well as cross-cutting risks that overlap with two or more of these categories.[22]

When mapping AI services and products against this framework, it is apparent that the risks to children that manifest are not new: rather they are amplified. As such, without considering children early in the design of these systems, there is a greater likelihood of their rights and needs being overlooked and/or severely impacted. The rapid development of AI systems in the last few years, and the tech industry’s continued mantra to ‘move fast and break things’ and to not “worry about things that can be fixed later”[23] risks trading the safety of children for greater profit and status.

Annex A[24] provides a summary of the rights most relevant to children and AI systems. Below, we expand upon some of the specific rights and how they intersect with the rapid development of AI systems and products.

Right to privacy

Article 16 of the UNCRC states that every child has the right to privacy.[25] In the digital world, this means that children’s data must not be over-collected or misused, for example for profiling, surveillance or to target advertisements towards them. Children must be offered high privacy by default, with their best interests observed at all times when processing their data.[26]

Almost every digital interaction is an exchange, underpinned by an invisible transaction in which the currency is data.[27] For children, access to real-world currency (e.g. cash) is often limited, meaning that the agency and control they have over their data is crucial to exercising their rights in the digital world.

AI challenges children’s right to privacy through use of pervasive data collection, scraping and processing, which can then be used to profile them – predicting or provoking their thoughts, feelings and actions to influence their behaviour.[28] AI systems that encourage children to share information and/or to build networks with others also undermines their right to privacy.[29] For example, Snapchat’s MyAI feature operates alongside Snapchat’s ‘quick add’ feature, which encourages children to expand their networks with others – including adults – with little friction.[30] Further, children have less developmental capacity for critical thinking and self-regulation and are more likely to take risks, making them more vulnerable to products designed to undermine their privacy for profit,[31] which also impacts children’s right to protection from commercial exploitation.

Right to life, development and survival

Article 6 of the UNCRC states every child has the inherent right to life.[32] In the digital world, including within AI systems, this means that children must be protected from risks posed by services and products that threaten their right to life, survival and development.[33] It also would be relevant in the decisions made about children in relation to their health, such as where AI is used identify risks of specific diagnoses.[34]

However, there are already examples of where this right has not been met.

Synthetic material created by AI can be used to abuse, humiliate and bully children, and facilitate crimes that directly impact children.[35] For example, data published by the Internet Watch Foundation (IWF) found that AI-generated child sexual abuse material (CSAM) rose 400% in the first six months of 2025.[36] This undermines both their right to life but also their right to be protected from sexual exploitation.[37]

The output of harmful content and activity from AI systems, notably companion chatbots, is thought to have already contributed to the tragic deaths of children.

In 2024, 14-year-old Sewell Setzer III took his own life. His mother said that, after becoming obsessed with a chatbot on Character.AI, Sewell developed an emotional attachment to the bot. The bot, which was named after a character from a popular TV show, gave him advice, but also allowed him to engage in romantic and sexual conversations.[38]

In March 2025, 16-year-old Adam Raine took his own life. His father found that Adam had been discussing ending his life for months before his death with ChatGPT, which supplied information about methods to end his own life.[39] Adam originally used ChatGPT for schoolwork, but was able to bypass filters intended to signpost to vulnerable users to support helplines.

The design of these chatbots, in particular their anthropomorphisation (the human-like traits programmed to produce human-like responses),[40] exploit vulnerabilities in children’s development. This is particularly the case for teenagers, who seek validation from peers, explore relationships and pursue greater autonomy in their digital experiences.[41]

Right to non-discrimination

Article 2 of the UNCRC states that children’s rights apply to all children without discrimination.[42] In the digital world, this means ensuring services and products do not treat children unfairly or discriminate against them, such as through automated processes, profiling or decision-making on unfairly obtained data concerning a child.[43]

Unfair outcomes occur when an AI system makes a decision that is wrong because it is discriminatory or inaccurate. Discrimination occurs when bias leads to unfair outcomes for people with similar characteristics. Inaccurate decisions may be based on poor data, incorrect assumptions of the AI model or, specifically for children, because they are assumed to be adults.[44] Biases can also be unintentionally embedded by developers too, for example in the qualities of the best candidate for a job.[45]

Children have already fallen victim to harm through unfair decisions made about them by AI systems.

In 2020, 39.1% of pupils’ A-level grades in England were downgraded as a result of a ‘mutant’ algorithm used to predict their results.[46] The algorithm in particular favoured private schools and impacted disadvantaged areas of the country the hardest.[47] Separately, research into machine-learning in the social care system found that, on average, if the model identifies a child at risk, it is wrong 6 out of 10 times. Further, machine-learning models missed 4 out of 5 children at risk.[48]

Further, children have also been subject to unfair decisions made by AI that, although not directly about them, have adversely impacted them.

Research published by Amnesty International[49] found that 60 AI models used for in the Danish social security system, designed to analyse data about childcare benefit recipients and flag potential cases of fraud, entrenched discrimination disproportionately targeted more marginalised groups (notably second generation migrants).[50] This led to 20,000 families being wrongly accused of fraud, forcing them to pay back thousands in childcare benefit they’d received through the payments.[51] The lack of oversight, transparency and access to remedy by the Danish government meant that families were left ruined following the outcomes produced by unfair automated decisions.

A similar scandal also happened in the Netherlands, in which parents and carers were accused of childcare benefit fraud after an algorithmic decision-making system was used to detect fraud.[52] This pushed tens of thousands of families – many of which were low-income or belonged to ethnic minority groups – into poverty as a result due to debts owed to the tax agency. Some victims committed suicide, and thousands of children were taken into foster care.[53]

Right to education

Article 28 and 29 of the UNCRC states that children have a right to an education that is accessible to all on the basis of capacity[54] and fulfils the development of a child’s personality, talents, mental and physical abilities to their fullest potential – preparing them for responsible life in a free society.[55]

In the digital world, this means that technologies used in educational settings must be accessible to all children, be appropriate for their education and not allow for the surveillance of children.[56]

In addition, questions arise about the objective of education and whether technologies deployed in the classroom fulfil children’s physical, mental and social development[57] – for example, how much human interaction may be needed to meet this right (something ‘AI tutors’ would not be able to replicate) and whether looking at a screen assists a child’s education. Further, 5Rights Youth Ambassadors tell us that, although they are excited about the prospects of AI, many are concerned about the effect it could have on their education and their ability to think critically and creatively.

Ultimately, schools are places that children should be safeguarded from risk to allow them to play, socialise and grow in an environment that supports their development whilst upholding their rights. Although schools are rapidly integrating AI technologies, a number of risks and opportunities can be observed regarding children’s right to education.

Increasingly, education technology (EdTech) products are integrating AI, including on large administrative services such as Google Workspace for Education (Google Classroom)[58] and Microsoft 365 Education (Microsoft Teams).[59] Research by the National Literacy Trust found 45.6% of young people aged 13-18 use generative AI (GenAI) tools weekly or more often for schoolwork, using these tools to improve vocabulary and support creative writing.[60]

However, research published by the 5Rights-LSE joint Digital Futures for Children Centre,[61] which analysed five case studies of AI in EdTech used in schools in the UK, found that – whilst there is limited evidence to suggest these tools may support children’s right to education (in addition to their right to play, expression and access to information) – these are countered by actively undermining children’s right to privacy, safety and protection from commercial exploitation. For example:

• Magic School AI[62] was found to enable tracking cookies to advertisers of pornography sites, for users as young as 12-years-old;[63]
• Grammarly[64] was found to allow the marketing platforms of companies such as Facebook, Microsoft and Google to use education data for commercial purposes;[65] and
• Character.AI, which can be used by teachers and students to create ‘characters’, was found to sometimes confuse fictional characters and real figures and provide contradictory information to children.[66]

GenAI services and products can only enhance children’s education if children are at the centre of its design, development, deployment and governance. Adopting a holistic, child rights approach when developing these products, by not exploiting children and prioritising in particular their right to education, would ensure that it respects the best interests of children and uphold their rights.[67]

Existing legal and regulatory framework

2. To what extent does the UK’s existing legal framework provide sufficient protections for human rights in relation to AI?

The UK’s existing legal and regulatory framework addresses certain aspects of AI, notably across online safety legislation and data protection regulation. Whilst taken together this addresses certain aspects of AI systems, it does not encompass all the different ways in which AI services and products may impact children’s rights.

Online Safety Act 2023

The UK’s Online Safety Act (OSA) passed into law in 2023, and places strict requirements (duties of care) on tech companies to make sure their services are made safe by design and provide children a higher standard of protection than adults.[68] As of July 2025, services in scope must apply the measures set out in Ofcom’s codes of practice[69][70] in order to be considered in line with the Act. This includes:[71]

• Implementing privacy-preserving age assurance: This ensures that children cannot access services or products not intended for them, whilst ensuring that their right to privacy is upheld when using these systems.
• Filtering out harmful content before it reaches children: This means that services cannot serve up pornography, suicide and self-harm and eating disorder content to children.
• Giving children high default privacy settings: This includes blocking contact between adult strangers and children by preventing them from directly messaging children, turning off location sharing by default for children and preventing children’s profiles being recommended to adult users.

Certain AI services are in scope of the OSA. In November 2024, Ofcom wrote to services detailing how the Act applies to GenAI and chatbots.[72]

Whilst user-controlled AI chatbots (such as those created and shared with other users) are in scope of the measures specifically designed to protect children from harm,[73] questions arise around whether chatbots integrated into other services regulated under the Act (e.g. Meta AI on Instagram) would be within scope of the measures. This is because, under the Act, service providers only have to apply measures in relation to user-generated content, and the output of service providers’ own AI is not considered user-generated content.[74] This gap means that certain AI chatbots may avoid application of measures designed to safeguard children.

Further, whilst the Act and the measures in Ofcom’s codes primarily seek to provide children a greater level of protection in order to protect them from harm and fulfil their right to life and survival, there are several risks from AI systems that go unaddressed under the legislation. Notably, this includes:

• Capture: As ‘persuasive design’ is not listed as a harm within the Act,[75] Ofcom’s measures do not address persuasive and addictive design strategies (such as notifications[76] and anthropomorphisation[77]) used by tech companies to keep children on services by exploiting their evolving capacities which means they cannot self-regulate in the same way as adults.[78]
• Protection from commercial exploitation: Similarly, as ‘financial harm’ is not listed as a harm within the Act,[79] Ofcom’s measures would not apply in cases where an AI chatbot provides poor financial advice to a child.
• Evolving capacities: Finally, Ofcom’s measures for protecting children applies only depending on whether a user is above or below 18 years of age,[80] meaning all children at this stage are treated as one homogenous group and the experience of a 7-year-old and a 17-year-old would be the same. Whilst Ofcom has signalled it will explore age-appropriate design in further iterations of its codes,[81] respect for children’s evolving capacities remain a crucial principle of the UNCRC that is not currently recognised in the code.

The Age Appropriate Design Code

In 2018, Parliament introduced the Age Appropriate Design Code (AADC)[82] under the Data Protection Act 2018, which sets out 15 standards organisations must abide by when handling children’s data, enforceable by the Information Commissioner’s Office (ICO). The AADC’s scope applies to services likely to be accessed by children,[83] meaning AI services and products that children use would need to apply its standards.

The 15 standards of the AADC reflect the fundamental rights children are afforded under the UNCRC and its General comment No. 25 (2021). Positioned within data protection regulation, its standards seek to provide children with a high level of privacy by design and default, including requiring services to conduct Data Protection Impact Assessments (DPIAs) to assess the risks to children that may arise from data processing,[84] set high privacy default settings[85] and only collect and retain the minimum amount of data possible from a child.[86]

However, standards of the Code also bring potential for children in addition to their right to privacy, including their right to safety, freedom from commercial exploitation, best interests and access to remedy.[87] These are highlighted by dedicated standards which prevent the detrimental use of data which impacts children’s physical or mental health and wellbeing,[88] and the Code’s overarching standard that all decisions are taken within the best interests of children – not commercial or business interests.[89]

Despite this framework providing a robust basis for protecting children’s rights, and its creation leading to numerous design changes by even the largest tech companies,[90] the ICO is yet to clearly discern how its standards apply in the age of AI.

For example, the mass-collection and scraping of children’s data used to build large language models (LLMs) that GenAI operates on would likely contradict the AADC’s standards on a child’s best interests, the detrimental use of data, data minimisation and profiling.[91] Whilst the ICO has produced guidance to suggest AI scraping must observe GDPR,[92] it tracks that therefore this should also be observed under the AADC – although this is not explicitly stated within the Code.

As such, it should also be expected that the ICO would take action to enforce the AADC where there is a violation of its standards by an AI service or product that poses a risk to children’s privacy and safety. However, as we noted in our response to the regulator’s 2024-25 strategy for the AADC,[93] despite the Code having been enforceable for (now) four years, the ICO is yet to issue a single fine in relation to it.

3. To what extent is the Government’s policy approach to deploying AI, expressed in its “AI Opportunities Action Plan” sufficiently robust in respect of safeguarding human rights?

In our view, the Government’s AI Opportunities Action Plan overlooks both the risks and potential that AI can have for children.[94]

Although the Government’s response[95] outlined its intention to accept all 50 recommendations, including the rapid integration of AI technologies into public services such as education, children are largely ignored in the Plan, including how these technologies could be made safer, age-appropriate and rights-respecting for them.

This is particularly concerning when one of the recommendations noted in the plan is to “move fast and learn things”[96] when piloting AI technologies in school – where children should expect their rights to be upheld and respected.

Researchers from Oxford University have highlighted that, although there is a growing consensus globally around AI ethical principles, there is little conversation about how to apply them for children.[97] By not recognising children within its guiding policy document for rolling out AI technologies, the Government has, so far, failed to consider the complex and individual needs of children and their long-term wellbeing.[98] This is despite the fact for many children AI has become a ubiquitous part of their daily lives, and that they will be expected to take up the jobs of the future in an AI-augmented economy.

Possible changes to the legal and regulatory framework

4. What would be needed in any future UK legislation to protect human rights?

• To what extent should the same human rights standards apply to private actors as public bodies when they use AI?
• To what extent might different kinds of AI technology require different regulatory approaches?

Children’s rights are universal and apply to all children regardless of background, political or religious belief or status.[99] States have legal obligations to ensure children’s rights are respected, protected and fulfilled – notably setting up a regulatory framework to ensure that businesses respect human rights (including children’s rights). Businesses have a corporate responsibility to protect children’s rights.[100] In order to ensure children’s rights are upheld everywhere, and their needs accounted for, both the public sector and private sector must adhere to standards set out in any future AI legislation.

As such, this means different regulatory approaches are not necessarily required to uphold children’s rights in all AI systems – although setting common criteria for the design of AI systems and their governance is. It is also important that outcomes which ensure that children are protected, their rights upheld, and their diverse needs accounted for take precedence in any form of regulation.

5Rights Children & AI Design Code,[101] which provides a model for what rights-respecting AI regulation could look like, applies to the implementation of different AI systems across different sectors. However, it requires developers of any AI system to consider nine criteria throughout its development. The introduction of UK legislation that reflects these common standards would provide a strong basis for regulating AI products in a way that respects children’s rights.

Below are the criteria AI developers must consider:[102]

• Developmentally appropriate: Ensure AI systems are designed and operated to account for children’s differing needs and vulnerabilities at different ages and stages of development, by design and default.
• Lawful: Confirm the service or product is compatible with relevant local, national, regional and international law, rules and regulations.
• Safe: Do not create or amplify risks to children’s physical, mental or emotional safety and their wellbeing – including privacy and security risks.
• Fair: Evaluate the model treats children and their data fairly, creating outcomes that are equitable and just.
• Reliable: Assess the AI system functions as expected, with performance and outcomes remaining robust over time, even in unexpected or harsh conditions or where atypical data is introduced.
• Provide redress: Ensure there are clear reporting and complaints systems to seek actionable and effective recourse and remedy where an AI system has caused concern or harm, with priority given to reports that relate to children.
• Transparent: Provide accessible information to help understanding of what an AI system does, accounting for the capacities and needs of children.
• Accountable: Establish a chain of human and organisational responsibility for the lifecycle of an AI product, making clear how conformity with the Code has been met.
• Uphold rights: Ensure AI systems uphold children’s rights under the UNCRC and General comment No. 25, prioritising their best interests and taking into account children’s voices and opinions.

5. Who should be held accountable for breaches of human rights resulting from uses of AI, and on what basis?

• Where in the process of developing, deploying and using AI technologies should liability arise?
• What additional measures, if any, are needed to ensure that individuals have sufficient redress where they have suffered harm because of the use of AI?

The responsibility applies to everyone across the value chain in the development of an AI system throughout the lifecycle of an AI product – from design and initial implementation to deployment and its eventual decommissioning.[103] It is worth noting, however, that these responsibilities might be different at certain stages (e.g. those designing the system would have greater obligations for transparency and explainability).

Regarding redress, as we note in Question 4[104] these mechanisms should be clear and provide priority to complaints related children. This means reporting systems must be age-appropriate, designed so that they are comprehensible, an appropriate length, clearly presented, and understandable to all children, regardless of age or background.[105] Further, redress must also be accessible – it must be easy for the final consumer (including children) to have redress without having to figure out the whole value chain of an AI system.

Children must be given information and supported throughout to understand how complaints are being actioned. Research commissioned by Ofcom[106] has found that, although children are aware of reporting and complaints systems, few actually do so due to a lack in trust of these complaints being actioned. People acting on behalf of children (e.g. parents, carers and representative organisation) must also be able to bring complaints where children’s rights are at risk.

6. How might regulation match the pace of AI technology, such as the emergence of agentic AI, to ensure human rights are preserved as technology continues to develop?

Regulation must be futureproof and establish broad principles and processes that any type of AI system must follow and respect. (See: Question 4).

7. How could regulation take account of the international nature of AI? How could it address the potential consequences for human rights in the UK of the malign use of AI by regimes in other countries?

The development of effective AI regulation requires a global consensus that these systems must be built with the best interests of consumers, children and their rights in mind. International agreements, such as the Global Digital Compact,[107] and the Council of Europe Convention on Artificial Intelligence and Human Rights, Democracy and the Rule of Law,[108] include clear obligations for states globally to adopt AI in a way that suits humanity and to strengthen legal and policy frameworks to protect and promote children’s rights.[109]

The UK must be a leader in driving these international frameworks and the Government must work with other nations globally to ensure that consumers and children’s interests take precedence over business interests. Further, in order to ensure that digital products are developed with children’s rights and needs in mind, it is crucial that standards, regulation and legislation are based in principles for the rights-respecting development of these technologies. By setting up a robust framework for the protection of human rights (including children’s rights), this could start similar processes in other countries – as with the UK’s Age Appropriate Design Code, which has since been exported globally.[110]

8. How much difference will the Council of Europe Framework Convention on Artificial Intelligence and Human Rights, Democracy and the Rule of Law make to the protection of human rights in the UK?

The Council of Europe Framework Convention on Artificial Intelligence, Human Rights, Democracy and the Rule of Law[111] is the first international treaty passed on AI, signed, but yet to be ratified by the UK. During its drafting, 5Rights worked closely as an observer to the committee.

The Convention is clear that the development of AI must respect human rights – including children’s rights. This obliges the UK to place the right legal framework to ensure that AI respects human rights, the rule of law and democracy. This framework should follow the general principles of human dignity and individual autonomy, transparency and oversight, accountability and responsibility, equality and non-discrimination, privacy and personal data protection, reliability and safe innovation.[112]

Specifically, Article 18 of the Convention is clear that signatories must take account of the specific vulnerabilities and needs of children.[113] Additionally, its explanatory report mentions the need to consider General comment No. 25 on children’s rights in relation to the digital environment[114] in the implementation of the Convention. This explicit recognition of children’s rights within the treaty represents a step forward in the international governance of AI systems, recognising that a distinct approach is required for children in the design of these technologies.

The Convention sets clearly the bar for the UK to ensure fundamental rights, including children’s rights, in relation to AI systems. However, it remains up to the UK to ensure proper implementation of the Convention. In that regard, it must ensure that its principles apply to the private sector, which the Convention allows states parties to decide whether to exclude from its scope.[115][116]

The UK Government should ensure it fully applies the Convention to the private sector in line with its other international commitments, sending a powerful statement to other signatories that children’s rights apply across all AI systems. This would send a powerful statement to other member states.

9. What lessons can be drawn from regulation of the impact of AI on human rights in other jurisdictions, such as the European Union?

The EU’s AI Act is a groundbreaking piece of legislation that is a major step forward for responsible development of AI systems – particularly when it comes to protecting children.[117]

The law provides a horizontal regulatory framework based on risk and sets a strong precedent by explicitly recognising children’s rights as outlined in the General comment No. 25.[118] It notably prohibits AI systems that “exploits any vulnerabilities of a natural person or a specific group of persons due to their age… with the objective, or the effect of, materially distorting the behaviour of that person or a person belonging to that group in a manner that causes or is reasonably likely to cause that person or another person significant harm.”[119]

The European Commission explicitly recognises that “age” refers to children and that the harms they face may be particularly severe and multifaceted. It therefore warrants a precautionary approach.[120] Additionally, the AI Act further provides that high risk systems must be subjected to a risk management system that should particularly account for the impacts faced by children.[121] It is worth highlighting that toys and AI systems used in educational settings are considered high risks under its Annex III.

The AI Act is underpinned by many secondary instruments, including codes of practices and technical standards. To ensure age appropriate and safe by design AI systems, it is crucial that those instruments embed children’s rights considerations. Their drafting must therefore ensure the effective consultation of affected stakeholders, including children’s representatives and experts organisations. The Act’s Code of Practice for General Purpose AI,[122] designed to instruct developers how to comply with their duties under the Act, has limited considerations of the systemic risks to children’s rights.

Whilst recognising the exacerbated risks and harms faced by children, the legislation could go further in consistently and comprehensively considering children across its measures. Children are entitled to and require distinct protections, and this must be clearly set out in any legislative framework. This is particularly pertinent given the unique vulnerabilities children have as they develop their physical, psychological, cognitive and emotional capabilities.

(Sept 2025)