Written evidence submitted by Dr. Mark Wong (DCG0029)

 

Co-Design with Minoritised Ethnic Communities for Trustworthy and Responsible AI in Government

Dr. Mark Wong, Senior Lecturer in Public Policy and Research Methods, Deputy Head of Urban Studies, University of Glasgow

This submission puts forward recommendations and evidence on “Digital Centre for Government”, which the Committee should address urgently as priorities.

Key Messages

Introduction

  1. Making Artificial Intelligence (AI) safer and more trustworthy is high on the global agenda, as the UK announces the AI Opportunities Action Plan with digitisation of public services prioritised, whilst the EU develops the first-in-the-world AI Act.

 

  1. But evidence shows there is growing public mistrust and fear about how AI is used in unfair ways in the private and public sectors. After the rapid acceleration of digitisation and use of algorithmic decision-making during the COVID-19 pandemic, there is an increased risks of AI harm on racially marginalised communities. This includes racial bias of AI used in governments, police, and healthcare and medical devices, which are disproportionately harming and discriminating against adversely racialised communities.
  2. The UK Government’s proposal to harness opportunities of AI is a significant moment to change the path for AI to benefit not only the few—not even just the majority—but one that truly benefits all and does no harm. We need AI that is fair and equitable “by default”. 

 

  1. To strengthen responsible AI, we must take a new approach that centres the voice, needs, and actions of people who are most vulnerable to the risks of AI racial bias. The consequence of not achieving this is dire. We have to question who have historically been left-behind, marginalised, and silenced in the past. To show bold leadership and get this right will put the UK at the forefront as the global leader in AI that the UK Government aspires to be.

 

  1. Recommendation 1: the Government should adopt participatory AI governance models, which center the needs and priorities of people who are most negatively impacted by AI, using co-design methods and principles.

 

  1. Co-design is a helpful methodological tool to ensure Minoritised Ethnic people have meaningful participation in making decisions on how AI is used and developed in the government. This can be high-level panel at the UK Government level, which governs and develop principles and decisions on how AI is used across UK government departments.

 

  1. The Minoritised Ethnic People’s Code of Practice for Equitable Digital Future, provides seven co-created principles to ensure digital services are designed in equitable ways, and is published by University of Glasgow and lead-authored by Dr. Mark Wong.

 

  1. The Government should develop a people’s panel, with members of the public who provides specialism and expertise by lived experience and/or learning about the harms of racial bias in AI and how AI affects people’s lives when used in a problematic way by different government sectors. This is often-called the co-design of AI or participatory approach to responsible AI innovation.

 

  1. In addition, the Government should create a robust UK evidence base to examine what marginalised communities, especially Minoritised Ethnic people, want and need AI to be used for (and what AI should not be used for), for what purpose, and what areas AI need to be safer and more ethical. This can be used to widen the evidence available to the people’s panel as well as the government to make more informed decisions about how AI is used and developed, and ultimately democratising the way that AI Is developed.

 

  1. Recommendation 2: Regulations are needed to ensure AI are developed together with people from adversely racialised communities right from the start, and not just retro-fixing problems after AI is deployed.

 

  1. The typical software development cycle assumes that technical problems should be rectified after the product is developed and is being tested. The way that AI is developed now, therefore, suffers from a lack of understanding and empathy of how its use impacts people. The AI development process is, at best, ambivalent towards how it can have adverse impacts on marginalised communities, particularly people who are adversely racialised. In many cases, it assumes that AI systems (and its underlying data) are neutral, when it in fact is not.

 

  1. Such neglect of the impact and outcomes of how AI is used needs a fundamental shift in the design process and lifecycle of AI innovation. We are now no longer in the realm of just testing things out and see how it progresses. We are in a world that our public transport, roads, and other everyday infrastructure has better safety standards. Whereas anything regarding “innovation” in the digital and AI is given a “free pass” in the sake of innovation. Even though we know many AI systems, especially automating decisions by algorithms, are likely to harm people who are adversely racialised and racially marginalised. We need to acknowledge, and build better support around, harmful conditions of AI that are not the same for everyone.

 

  1. To talk about responsible AI without challenging its path towards the future will merely allow AI systems to perpetuate inequalities and social injustice, such as systemic racism. The UK AI ecosystem needs to be more honest and transparent about the harms and discrimination that AI is causing to those who are most adversely impacted and, importantly, how AI and data systems can perpetuate inequalities and systemic racism, especially in the access, outcomes, and experiences of service that are powered by AI.

 

  1. Recommendation 3: The Government must ensure and regulate that people’s voices and priorities are centred at the AI development process when developing, using, and evaluating AI in government and public sector.

 

  1. This approach is termed as human-centred AI. So people’s needs are at the centre of making decisions about AI uses and what purpose it serves, instead of technical convenience or cost issues. Members of the public should be included to determine and prioritise the problems that need solving, the how and why, as well as the what.

 

  1. People from adversely racialised communities should be given power in the decision-making on when AI should or should not be used. More importantly, it is crucial to provide meaningful participation for people that the AI will have the most adverse impact on. These communities should determine how insights or predictive analytics are being interpreted, about themselves and predict their behaviours, to help avoid uses of AI causing harm.

 

  1. One way to develop safer AI is to open up the development process to involve adversely racialised/Minoritised Ethnic communities as early and iteratively as possible. AI and data science communities often talk about developing technical solutions and data systems that reflect “ground truth” (i.e. a some-what absolute standard of what is happening in real-life). But what is less understood is that the “ground truth” itself is a mirror of the systemic inequalities in society. Employing AI and algorithms to automate decisions based on such “ground truth”, will only perpetuate and replicate these inequalities.

 

  1. AI innovation must take into account the disproportionate harm that use of AI can cause to racially marginalised groups in society. These problems can be foreseen and understood, particularly from the perspective of the people who are most impacted, before AI is being developed and deployed. But the relevant lived experience and expertise in structural factors, such as systemic racism, must be brought in to unpack these issues, when AI is used in a public sector context.

 

Conclusion

  1. A more inclusive, responsible way of developing AI has in fact been called for by over a decade by many international researchers and activists.

 

  1. There are several recent successful examples in Scotland, such as citizen panels, youth panels, co-design workshops, and an Anti-Racism Interim Governance Group, who are putting people and communities together with experts in systemic inequalities to determine a fairer future in Scotland’s AI and data landscape. There is also learning from the approach that the Scottish Government and Scottish AI took in development of the Scotland’s AI Strategy: Trustworthy, Ethical and Inclusive.

 

  1. In the current AI ecosystem, how AI is developed and what purpose it served is often decided behind closed doors and, therefore, is exclusivist. The UK Government should introduce urgent measures of national intervention and participatory governance of AI, such as co-design and people’s panels. Evidence supports high-level intervention to enact change is the most effective intervention to ensure AI is developed in a trustworthy, responsible, and fair manner. We need a new approach to AI that centres the needs and voices of the people that it’s meant to serve.

 

03 March 2025

 

Author Bio

Dr. Mark Wong is Senior Lecturer/Associate Professor in public policy and research methods and Deputy Head of Urban Studies and at the University of Glasgow. He is an expert in: digital society and policy, racial bias in AI, and responsible AI and data. He has a PhD in social policy from the University of Edinburgh and has a track record of publishing in top peer-review journals and winning national research grants funded by the UKRI.

He is a Scottish-Minister appointed member of the Scottish Government’s "Interim Governance Group on Developing an Anti-Racist Infrastructure”. He has also served in several policy advisory roles for the Scottish Government, NHS, and Public Health Scotland, addressing issues of systemic racism in data infrastructure at a national level in Scotland. 

 

Dr. Wong’s research addresses the bias and harms of AI and data on Minoritised Ethnic. He has strong experience in promoting responsible, equitable, and sustainable use of AI, especially in Scotland's policy landscape. He is leading innovation in participatory and co-design approaches in the development of AI and algorithms, focusing on "ethical by design" and marginalised communities' participation in AI innovation.

 

Bibliography / Resources: