WRITTEN EVIDENCE SUBMITTED BT GROUP

(GAI0091)

The UK Artificial Intelligence (AI) ecosystem has made major progress in developing and using trustworthy AI. The adoption of a proportionate AI governance framework is however necessary to further foster public trust and enable the UK to secure the full benefits of AI while mitigating its possible risks.

As the number one patent filer in the UK[1], BT Group has been carefully looking at innovating in AI in a responsible way through our AI research program and our dedicated AI and Data Solutions team within BT’s new Digital Unit[2]. Our goal is to be the most trusted connector of people, devices and machines in the world by 2030[3]. We have adopted responsible tech principles as well as internal check and balances (including training) to ensure we develop, use, buy and sell AI products and services in a way that secures the benefits and minimises the potential harms.

Based on this experience, we understand that building a robust but proportionate AI governance framework involves complex trade-offs. We therefore welcome the opportunity to input into the Science and Technology Committee inquiry on AI governance and offer below BT Group views on to achieve the best balance between oversight and room for innovation.

If you would like us to provide any further information regarding our response, including our tech principles and internal governance framework related to AI, please contact daniel.4.wilson@bt.com and garance.hadjidj@bt.com             

How effective is current governance of AI in the UK? What are the current strengths and weaknesses of current arrangements, including for research?

Regulators, and particularly the ICO, have risen to the challenge of adapting to Artificial Intelligence (AI) and provided useful guidance. We however agree with the challenges identified in the AI policy paper published by the UK Government: a lack of clarity of relevant regulatory mechanisms; overlapping regulations and laws; inconsistent approaches across regulators; gaps in some sectors where AI has not yet been the subject of regulatory scrutiny; and insufficient regulators skills.

A good governance of AI requires a balanced mix of rights, obligations and ethics that will allow innovation to flourish responsibly. We therefore support the proposal of the UK Government in its AI policy paper that focus on:

• the risk-based approach to regulating AI which would ensure leeway for inventions by businesses that are cutting-edge but harmless, and thus increase public trust and adoption
• the light-touch regulatory approach which would empower UK regulators to rely on their expertise to develop fit-for-purpose guidance on ‘what is possible and should be done’ and thus enable businesses to innovate safely.
• the adoption of value-based principles well rooted into ethical principles and aligned with the OECD work that would help the UK be among the world leaders in developing responsible AI applications

Good governance of AI can’t however be achieved in isolation. We believe Government should therefore conduct and publish a review of existing policy, guidance, standards and regulation relating to AI governance across sectors and international jurisdictions. The exercise would be particularly useful for companies that might not have the resources to track international developments but that want to scale-up internationally.

What measures could make the use of AI more transparent and explainable to the public?

To make the use of AI more transparent and explainable, policymakers need to:

• provide guidance to regulators of how the principles of ‘transparency’ and ‘explain ability’ could be interpreted so we have better consistency across domains. This will contribute to increase legal certainty for businesses and facilitate the public understanding.
• introduce guidance that point businesses to use cases highlighting best practices. This way businesses will know the ‘art of possible’ while being able to pick the practices that work for them, avoiding the creation of bottleneck in internal processes and mechanisms already in place. 
• develop concrete auditing guidelines which evolve in tandem with standards and best practice as they develop.  Note that for an auditing regime to be trusted, there must be some form of accreditation assigned to the auditors.  We think there is a role for regulators to identify organisations with sufficient understanding of the technology and principles, and who are able to use the audit tools in an effective and responsible way.
• assess the opportunity to introduce an AI Incident Database at the UK level to support the AI ecosystem in learning from mistakes and near misses across industry (see Welcome to the Artificial Intelligence Incident Database). The database should also include the remedies adopted so efficient solutions are better communicated.
• run award campaigns for compliance programs so guidance is promoted and leaves a mark on the wider ecosystem.

Those measures will contribute to better consistent AI model documentation and increase AI transparency and explainability, and ultimately public trust in AI. Indeed, currently there is a variety of formats available (Google’s Model Cards, IBM’s AI Fact Sheets, TM Forum’s AI Model Data Sheets) but there is little agreement in what information should be captured and how to do this.

At a general level, public mistrust of AI’s development and use comes mainly from the lack of public awareness of AI and a predominantly negative narrative around AI largely coupled with a recurrent error of presenting AI as sentient and something that develops itself. Based on BT research on AI careers as well as BT/CogX’s AI Ready Nation event which took place on 24th October 2022[4], we believe there is significant potential to increase the public’s understanding of Artificial Intelligence, with potential policy interventions including:

• a compulsory AI awareness course for 14-18 years old children at school so no child can leave school without having a basic understanding of AI:
• a national campaign to debunk myths around AI and AI careers. In Canada, to boost public awareness and foster trust, they created the Open Dialogue program on AI which was based on regional workshops to consider and discuss the ethical dilemmas posed by particular AI applications. In Singapore, the Government created the ‘Lab on Wheels’ which introduces AI and other emerging technologies by bringing technology-interactive showcases to schools and the community.

How should decisions involving AI be reviewed and scrutinised in both public and private sectors? Are current options for challenging the use of AI adequate and, if not, how can they be improved?

In our view, it is right to adopt a risk-based approach to regulating AI since it will help avoid unnecessary regulatory red tape and wasting of resources on uses of AI that are low risk, thus increasing public trust. To meet this two-fold objective, further clarity will be required as to the meaning of a regime that focuses on “applications of AI that result in real, identifiable, unacceptable levels of risk, rather than seeking to impose controls on uses of AI that pose low or hypothetical risk”. Indeed, considering the risks posed by AI applications solely within particular sectors may lead to gaps in understanding.

We therefore suggest the introduction of a common framework which will explain the parameters for regulators to assess whether the risk arising from AI application is ‘real’ and ‘unacceptable’. Such a framework will increase transparency and encourage better scrutiny. For example, regulators should be looking at:

• Decision: the outcome of the AI system should be taken into account when assessing if the context and use is indeed high-risk. AI based systems taking autonomous decisions about, for instance, which customer service agent an inquiry from a customer should be redirected to, do not pose a risk to health, safety or fundamental rights.
• Built quality: it should be taken into consideration how the AI systems has been built and how well the built process has been documented. For example, has the data used in the machine learning process been tested for quality and bias, is the data going into the AI system for decision making undergoing the same quality and bias tests, has the decision performance of the AI system been tested against several state-of-the-art metrics, is there an integrated approach to continuous monitoring and testing after deployment, is there a procedure for detecting outlier decisions and a process to mitigate against misleading decisions?
• Transparency: the less transparency, the more difficult it is to monitor or control the AI system, and as such the risk of using the system is higher. Such parameters could be used to precisely define not only high-risk applications of AI, but also those contexts and use-cases where the risk is especially relevant. The publication of such framework with the parameters and use cases will be beneficial for both regulators and industry, especially SMEs.

The construction of this framework should be:

• informed by business practices and real-life experience of communities susceptible to be impacted.
• periodically reviewed based on regular exchange with the wider ecosystem to guarantee that it won’t hinder market entry and innovation as well taking into account the evolutionary path of the technology and give enough time for businesses to adapt to changes.
• informed by the pros and cons of different international examples and ensuring sufficient alignment particularly with the EU - so businesses operating across jurisdictions, including start-ups who are expanding, aren’t facing too much red tape.

Finally, for an efficient oversight of AI decisions, a holistic upskilling strategy, for regulators, businesses and the UK’s wider population, is critically needed since the adoption of responsible AI won’t happen if the UK as a whole isn’t AI ready. This strategy should be a cross-pollination effort between Government, regulators (cf. CMA’s concrete experience in building its own data science team would be beneficial for others), industry[5], academia and non-profit as well demonstrated by the Singapore model[6]. 

For regulators, especially those who have not been engaged with emerging technologies, acquiring the right level of technical know-how as well as keeping up with technology developments, will pose significant difficulties. Different routes can be envisaged – from secondments of regulators to industry, to business/academia-led bootcamps for regulators, creating a talent pool or embedding AI experts into each regulator as done with Chief Scientific Officers. All have merits and could be combined as Government/Regulators test them as part of an iterative process nurtured by regular contact with the wider ecosystem. Finally, upskilling initiatives for regulators shouldn’t be limited to the expansion of technology knowledge but should extend to better understanding of commercial aspects as well as ethics and human-rights norms.

For the wider UK population, the UK should look at scaled-up solutions to make the country AI ready like Singapore or Finland.

How should the use of AI be regulated, and which body or bodies should provide regulatory oversight?

BT supports a light-touch regulatory approach which would empower UK regulators to rely on their expertise to develop fit-for-purpose guidance on ‘what is possible and what should be done’, thus enabling businesses to innovate safely. However, this approach entails some challenges and risks that will require a strong coordination between regulators. For this reason, we welcome the Government’s ambition to ‘seek to ensure that organisations do not have to navigate multiple sets of guidance from multiple regulators all addressing the same principle’. Delivery against this will be particularly critical for companies which are seeking to innovate, like BT is, in developing AI products and services across sectors or domains. In such cases, divergent approaches between regulators could stifle innovation.  To ensure coordination, we would recommend:

• an enhanced Digital Regulatory Coordination Forum (DRCF) to facilitate co-ordination amongst regulators - like the UK Regulators Network, UKRN - but with an obligation for regulators to engage with the forum. Indeed, the DRCF has been a welcome step change in joint working among member regulators; however, its voluntary nature has led to some missed opportunities for coordination. For example, DRCF could have facilitated greater co-ordination between the ICO and Ofcom during the development of the Data Protection and Digital Information Bill. The Bill introduces a new obligation on communication providers to report on suspicious or unlawful direct marketing activities such as nuisance calls to the ICO. The telecoms industry is already working closely with Ofcom to identify and prevent such calls, and rather than potentially create unnecessary or duplicative burdens with a separate regulator, the DRCF might have played a more significant role in facilitating data sharing between Ofcom and ICO, in lieu of creating a separate duty on industry.

• an enforcement / decision body if industry thinks decisions taken in different sectors are unjustifiably different. The Competition Market Authority (CMA) could be well-placed to play a role as a backstop in policing co-ordination and presumably opining if there should be a common approach or not. The CMA would indeed present several advantages from experience in building up serious tech expertise (e.g. new Data, Technology and Analytics Unit) to its ability to command action in some areas of economic activity without a sector-specific regulator and where the use of AI could be less scrutinised.

To what extent is the legal framework for the use of AI, especially in making decisions, fit for purpose? Is more legislation or better guidance required?

We believe the UK Government’s proposed approach to AI regulation is heading towards striking an appropriate balance between unleashing innovation to accelerate the growth of opportunities associated with low-risk AI and the need to put in place a framework that protect the users from potential dangers that high-risk AI systems may pose.

To make it fully fit for purpose, we would recommend:

• Further clarify the scope of the regulatory framework to ensure that not every single minor process could be considered AI. 
• Create a framework of risk to help regulators determine whether AI used within their sector or domain requires regulatory oversight, clarifying what is meant by ‘real and unacceptable risk’. 
• Create an outcome-based framework around the cross-sectoral principles with a compendium of use cases and best practices highlighting the ‘art of the possible’ to businesses in a practical and actionable way. 
• Stay closely engaged with the creation and adoption of international AI standards, which will be playing a major role in regulatory compliance in other jurisdictions. 

What lessons, if any, can the UK learn from other countries on AI governance?

Based on other countries experience, BT would recommend the following to measure to develop a world-leading AI governance that works for all parts of the AI ecosystem:

• the introduction of a voluntary self-assessment tool, like the one developed in Singapore, would be useful. This tool should be offered as a ‘north star’ for SMEs to ask the right questions and facilitate the creation of the right processes, and it should be voluntary only.
• running regulatory sandboxes to encourage developers to try their solutions within well-defined bounds without running the risk of regulatory action. Being able to address the risk of regulatory action is of particular concern to smaller companies with fewer resources. AI solutions applied in risky contexts that aim for built-in mitigation would also benefit from sandboxes and can help the industry learn how to best mitigate AI risk.

(November 2022)