Written evidence submitted by Dr. Daniele D'Alvia and Prof. Rosa Lastra
B. AI ADOPTION IN FINANCIAL SERVICES
B.1 Current Applications, and Future Trends
B.2 Comparative Analysis of AI Adoption in Financial Services
C. AI AND PRODUCTIVITY IN FINANCIAL SERVICES
C.1 Efficiency gains and transactional benefits
D. AI AND FINANCIAL STABILITY RISKS
D.2 Third-party dependencies, and algorithmic complexity
D.3 GenAI Hallucination, and Herding Behaviour
E. AI, CONSUMERS, AND VULNERABLE GROUPS
F. REGULATORY, AND POLICY RECCOMENDATIONS
F.1 Does the UK need new AI financial regulation?
Artificial Intelligence (AI) is transforming financial services at an unprecedent pace. From retail banking to algorithmic trading and risk assessment, AI is reshaping traditional financial models, improving efficiency, and increasing access to financial products. However, this evolution presents signficant risks to financial stability, consumer protection, and regulatory compliance. This opinion explores the impact of AI on financial services and provides a detailed analysis of the regulatory challenges and policy solutions necessary to ensure a balanced approach to AI adoption in the financial sector. Indeed, the deployment of AI in financial services must be carefully evaluated to ensure a balanced approach that fosters innovation while mitigating potential threats to systemic stability.
AI is currently utilised across multiple sectors within financial services, and its adoption varies signficantly among different industry segments:
Fintech firms, due to their agile business models and reliance on digital infrastructures, have demonstrated a higher propensity for AI adoption compared to traditional banks and traditional financial institutions (Arner, Barberis and Buckley, 2017). It is estimated that 70-80% of global trading volumes are now driven by AI algorithms (Loh, 2019). This raises concerns over market volatility, regulatory oversight, and ethical considerations. Indeed, proper practice of ethics requires three elements: a moral agent, reason and action. Ethics in finance is applying moral norms to financial activity broadly conceived (Russo, Lastra, Blair, 2019), and shall be adapted to AI models in particular.
The rapid adoption of AI in financial services is driven by several factors, including advacements in machine learning, data analytics, and cloud computing. Given their agility, Fintech firms are better positioned to leverage AI for optimising customer onboarding, streamlining regulatory compliance, and enhancing fraud prevention strategies.
The US and China have embraced AI in financial services at an accelerated pace, benefiting from significant investments in AI research and regulatory sandboxes designed to promote financial innovation.
Challenges such as regulatory fragmentation or regulatory uncertainty, third-party dependencies, and data silos and governance as well as legacy IT systems that hinder seamless AI integration remain barriers to maximizing AI’s potential (see further point in C.2 below). Specifically, the UK’s financial sector has positioned itself as a global leader in AI adoption, benefiting from a well-developed financial ecosystem, and access to highly skilled workforce.
AI enhances efficiency by automating routine tasks, improving risk assessment, and reducing operational costs (Gai, Qiu and Sun, 2018). AI has the potential to improve productivity in financial services by automatic routine tasks, enhancing decision-making processes, and reducing operational costs. Financial institutions are increasingly deploying AI in areas such as:
Such key areas are targeted to have greater efficienty and accuracy. Indeed, as argued by Abraham (1986), accuracy is the main parameter to provide a correct risk assessment that can be achieved only through the correct distribution of risks and classification, albeit the main hurdle to such a result is the difficulty involved in identifyin homogenous instances in a risk class as well as anticipating all the possible “private risks”. Our own ignorance and our awareness of the indterminateness of the instances and variables that exist in the world remains an important obstacle, and can lead to a reduction in accuracy. The resulting unknoability of the instances justifies the need for classifcation and distribution of risks, but at the same time the impossibility of predicting all the variables reduces homogenity of classes. If any risk class were absolutely homogenous, there would be no probability in the results, and the idea of probability becomes meaningless. Hence, a perfect level of accuracy is by definition impossible to be achieved due to our inherent erroneous instinct and perception that necessarily are either directly or indirectly reflected in the alghorithms through which we design AI models and the data we use to inform such processes.
Despite these benefits, several challenges hinder widespread adoption of AI in financial services. Regulatory uncertainty remains a major barrier, as financial institutions struggle to navigate the complex and evolving regulatory landscape governing AI applications. Data quality and accessability issues further complicate AI implementation, as many financial institutions operate within fragmented data ecosystems. Moreover, legacy IT infrastructure presents a signficant hurdle, limiting the seamless integration of AI-powered solutions.
While AI enhances efficiency, concerns over job displacement persist, particularly in bank-office functions and customer service roles (Frey and Osborne, 2017). However, AI is also creating new job opportunities in AI governance, cybersecurity risk managment, and algorithmic auditing. The workforce transition necessitated by AI adoption requires upskilling initiatives and regulatory support to ensure a balanced labor market adapatation.
AI presents several risks to financial stability that requires proactive regulatory intervention (Danielsson, Macrae, Uthenmann, 2022). One of the most pressing concerns is the potential for AI-driven systemic risks in investment markets. The recent case of DeepSeek vs. Nvidia highlights the vulnerabilities associated with AI-driven investment strategies. The over-reliance on AI models for investment decision-making can lead to market distortions, unintended herding behaviour, and increased market volatility. In the DeepSeek case, AI-driven investment models fueled excessive speculation in Nvidia stocks, amplifying market momentum and creating a fragile investment cylce susceptible to sudden reversals. Such episodes underscore the need for robust regulatory mechanisms to mitigate AI-induced financial instability.[1]
The integration of AI in algorithmic trading has introduced risks associated with rapid market movements and price anomalies. The Flash Crash of 2010 (Kirilenko et. al., 2017) is an early but relevant example of how AI-powered trading algorithmms can exacertabe market instablity when feedback loops amplify sell-offs in a matter of seconds. Similarly, AI models relying on real-time sentiment analysis from social media and news feeds can be susceptible to misinformation, leading to erratic trading patterns and unexpected liquidity shortgates. AI’s ability to execute trades based on patterns in high-frquency trading environments presents unique challenges to traditiona risk management frameworks. Unlike human traders, AI does not engage in qualitative rasoning or contextual analysis, increasing the risk of automated decision-making amplifying systemic shocks (Danielsson, Macrae, Uthenmann, 2022).
The potential for cyber threats also heightens financial stability risks, as AI-powered attacks become increasingly sophisticated. AI-enhanced cyberattacks can manipulate trading algorithms, exploit vulnerabilities in automated lending models, or disrupt entire financial networks. These risks are particularly pronounced as financial institutions increase the reliance on third-party AI providers for risk assessment, fraud detention, and decision-making processes. The growing dependence on a small number of AI providers including Goolge, Microsoft, and Amazon, raises systemic concerns regarding concentration of power within financial markets. The risk of AI monopolization in financil infrastructure could lead to a scenario where a failure in one major AI system has cascading effects across multiple financial institutions.
The complexity and opacity of AI models present additional concerns for financial stability (Danielsson, Macrae, Uthenmann, 2022). Many AI-driven financial models operate as “black boxes”, making it difficult for regulators and financial institutions to undertand the reasoning behind AI-generated outcomes. The phenomenon of AI hallucination, where AI models generate misleading or incorrect outputs, poses a significant challenge to regulatory oversight. If AI models produce erroneous credit risk assessments or flawed investment recommendations, the financial sector could face unanticipated risks, undermining confidence in AI-driven decision-making. The absence of standardised interpretability mechanisms exacerbates this issue, as regulators struggle to impose transparency requirements on opaque AI models.
AI’s role in reinforcing systemic risk is further compounded by the emergence of “herding behaviour” among AI-driven financial models. When multiple AI systems operate on similar data inputs and market signals, they may converge on identical investment strategies, amplyfying market volatility. This dynamic was evident in the DeepSeek vs. Nvidia case, where AI-driven models simultaneously fueled speculative investment (D’Alvia, 2023), leading to an unsustainable asset bubble. The concentration of AI-powered investment strategies increases the risk synchronised market movements, reducing market resilience to external shocks.
As AI-driven financial models become more complex, the need for robust regulatory safeguards becomes paramount. Regulatory authorities must develop frameworks for AI oversight that address systemic risk concerns while preserving the efficinecy gains AI offers. Given AI’s growing influence on financial markets, the development of AI stress-testing mechanisms should be a priority for regulatory bodies. These mechanisms should assess the potential consequences of AI-driven trading failures, cyberattacks, and algorithmic misinterpretation in high-frequency trading environemnts. Additionally, regulatory bodies must enhance transparency requirements by mandating algorithmic explainability standards, ensuring that financial institutions can justify AI-generated decision-making processes.
In other words, while AI offers vast potential, we must ensure that it serves the real economy and does not become a tool for unchecked speculation or systemic risk. Nonetheless, in the absence of a legal definition of speculation, the systemic effects of the “Magnificent 7” in New York, it is an important signal that shall remind us that we must never be complacent in our assumptions that we can simply regulate all market dangers away. Furthermore the pervasive influence of algorithms goes beyond financial concerns and it directly touches upon culture and society. Altorighms extend beyond cultural recommendations, affecting critical areas like real estate, healthcare, and legal decisions, often perpetuating existing biases and inequalities. Brown suggests that focusing solely on algorithms might divert attention from broader systemic challenges that require substantial societal changes.[2]
AI-driven cyberattaks pose systemic risk to financial institutions (Bouveret, 2018). AI can both enhance cybersecurity and be weaponised to exploit vulnerabilities in digital banking infrastructures. Specifically, AI-driven cyberattacks have the potential to compromise sensitive financial data, disrupt digital banking infrastructure, and undermine public trust in financial institutions.
The increasing reliance on third-party AI providers raises concerns about concentration of risk. The financial market’s dependence on a small number of AI firms, such as Google, Microsoft, and Amazon, increases systemic vulnerabilities.
The complexity of AI models introduces new risks, as financial institutions struggle to interpret and validate AI-generated outcomes. This can create the potential for herding behaviour in algorithmic trading, which in turn increases financial instability risks.
The deployment of AI in financial services offers several benefits to consumers, including personalised financial products, improved fraud detection, and enhanced access to credit. AI-powered chatbots and virtual assistants are enabling more efficient customer service interactions, while AI-driven analytics are helping financial institutions identify and assist vulnerable outcomes. However, AI also presents risk related to embedded bais, data privacy, and transparency.
One of the major concerns in the risk of AI models reinforcing existing biases in financial decision-making. AI-driven credit scoring models, for instance, may inadvertently discriminate against certain demographic groups if trained on biased datasets.[3] The opacity of AI decision-making further exacertbates concerns over accountability, as consumers may struggle to challenge unfair AI-generated outcomes. Ensuring transparency and fairness in AI applications is crucial to protecting consumer rights and maintaning trust in financial services.
AI models can perpetuate or amplify biases,[4] leading to discriminatory lending practices (Berk et. al., 2021). Lack of transparency in AI decision-making undermines consumer trust as well as stricter data-sharing frameworks are needed to enhance AI effectiveness while safeguarding privacy. Indeed, data protection is another critical issue, as AI systems require extensive data inputs to function effectively. The sharing of consumer financial data raises questions about privacy, consent, and security. While regulatory frameworks such as the UK General Data Protection Regulation (UK GDPR) provide important safeguards, further legislative developments may be necessary to address the unique challenges posed by AI in financial services.
The rapid evolution of AI necessitates a dynamic and adaptive regulatory framework. Regulators must strike a balance between fostering AI-driven innovation and safeguarding financial stability and consumer rights. Introducting AI-specific financial regulations, mandating alghorithmic auditing, and enhancing cybersecurity resilience should be key priorities for policymakers. Financial regulators must also enhance their AI oversight capabilities by investing an AI-driven tools, fostering regulatory sandboxes, and strengthening public-private partnerships for AI goverannce.
The regulatory landscape for AI in financial services varies signficantly across jurisdictions, with the European Union, the United States, and the United Kingdom diverging in their approaches. To this end, we argue that the European Union has taken a proactive stance with the proposed AI Act (Regulation (EU) 2024/1689), which aims to introduce a risk-based framework for AI regulation. This legislation would like to harmonise rules on artificial intelligence among the EU Member States. The aim of the rules is to foster trutworthy AI in Europe. Specifically, the AI Act is part of a wider package of policy measures to enhance the development of trustworthy AI models, which also includes the AI Innovation Package,[5] the AI Factories,[6] and the Coordinated Plan on AI.[7] Those measures guaranteee safety, fundamental rights and human-centric AI, and strengthen uptake, investment and innovation in AI across the European Union. To thie end, categorises have been elaborated to distinguish AI applications into high-risk, limited-risk, and minimal-risk categories, imposing strict compliance obligations on financial institutions deploying high-risk AI models. The EU approach emphasizes transparency, accountability, and ethical AI deployment, requiring organisations to conduct impact assessments, ensure explainability, and mitigate bias in AI-driven decision-making processes. By contrast, the United Sates has adopted a more fragmented and market-driven approach, relying on sector-specific regulations and self-regulation by technology firms. While agencies such as the Securities and Exchange Commission (SEC) and the Commodity Futures Trading Commission (CFTC) oversee AI-driven trading practies, there is no overreaching AI regulatory framework comparable to the EU AI Act. The US approach prioritizes innovation and comptetitiveness, alowing financial institutions greater flexibility in AI adoption but raising concerns over regulatory gaps and consumer protection.
The UK, through the Financial Conduct Authority and the Prudential Regulation Authority, has sought to balance innovation with robust oversight. The FCA has emphasized the importance of AI transparency, fairness, and acocuntability, launching initiatives such as regulatory sandboxes and AI-focused guidance for financial firms.[8] However, as AI adoption accelerates, there is an urgent need for a comprehensive regulatory framework tailored to financial servces, ensuring that the UK remains a global leader in AI-driven finance while safeguarding financial stability and consumer rights. Specifically – we argue – that the UK’s approach to AI regulation should emphasize transparency, accountability, and ethical considerations. Strenghtening AI ethics guidlines, mandating explainability in AI decision-making, and imposing strict data protection measures will be essential to ensuring responsible AI deployment in financial services. As AI continues to shape the future of finance, a proactive and risk-based regulatory approach modelled on the EU AI strategy will be crucial in mitigiating systemic risks while enabling AI to drive financial innovation.
The existing regulatory framework based on a twin-peaks model between the Financial Conduct Authority and the Prudential Regulation Authority covers AI indirectly, but lacks specific AI-focused provisions. Regulatory sandboxes, such as the FCA’s Innovation Hub, should be expanded to test AI models before deployment at scale.
To this end, the UK Government should invest in:
February 2025
Abraham K. S., Distributing Risk: Insurance, Legal Theory and Public Policy (Yale University 1986)
Arner D., Barberis, J., and Buckley, R., “FinTech, RegTech, and the Reconceptualisation of Financial Regulation” (2017) 37 (3) Northwestern Journal of International Law and Business, 371 – 413.
Berk, R., Heidari, H., Jabbari, S., Kearns, M., and Roth, A., “Fairness in Criminal Justice Risk Assessments: the State of the Art” (2021) 50 (1) Sociological Methods and Reseearch, 3 – 44.
Bouveret, A., “Cyber Risk for the Financial Sector: A framework for quantitative assessment” (2018) IMF Working Paper.
Danielsson, J., Macrae, R., Uthemann, A., “Artificial Intelligence and Systemic Risk” (2022) 140 Journal of Banking and Finance, 1- 9.
D’Alvia, D., The Speculator of Financial Markets: How Financial Innovation and Supervision Made the Modern World (Palgrave Macmillan Studies in Banking and Financial Institutions 2023).
Frey, C., Osborne, M., “The future of employement: how susceptible are jobs to computerisation?” (2017) 114 Technological Forecasting and Social Change, 254 – 280.
Kirilenko, A., Kyle, A., Samadi, M., and Tuzun, T., “The flash crash: the impact of high-frequency trading on an electronic market” (2017) 72 (3) Journal of Finance, 967 – 998.
Loh, H., “The rise of AI in financial trading: opportunities and challenges” (2019) 54 (2) Financial Review, 231 – 256.
Russo, C., Lastra, R., Blair, W., Research Handbook on Law and Ethics in Banking and Finance (Edward Elgar 2019).
[1] Daniele D’Alvia, “DeepSeek, Nvidia and the AI race that’s shaping the future” (31 January 2025) The Conversation UK, available at https://theconversation.com/deepseek-nvidia-and-the-ai-race-thats-shaping-the-future-248626, accessed on 24 February 2025.
[2] Brown T. M., “The technology that actually runs our world: the most dominant algorithms aren’t the ones choosing what songs Spotify serves you” (16 December 2024) The Atlantic, available at https://www.theatlantic.com/culture/archive/2024/12/cultural-algorithms/680987/?utm_source=facebook&utm_medium=cr&utm_campaign=2024_Content_InternationalTest_Prospecting_Sales_Standard&utm_content=121724_TechnologyRunsWorld_NA_NA_NoCTA, accessed on 28 February 2025.
[3] Financial Conduct Authority, “A literature Review on Bias in Supervised Machine Learning” (11 December 2024), available at https://www.fca.org.uk/publication/research-notes/literature-review-bias-in-supervised-machine-learning.pdf, accessed on 27 February 2025.
[4] Financial Conduct Authority, “Research Note: A Pilot Study into Bias in Natural Language Processing” (9 January 2025), available at https://www.fca.org.uk/publication/research-notes/pilot-study-bias-natural-language-processing.pdf, accessed on 27 February 2025.
[5] European Commission, “Commission launches AI innovation package to support Artificial Intelligence startups and SMEs” (24 June 2024), available at https://ec.europa.eu/commission/presscorner/detail/en/ip_24_383, accessed on 23 February 2025.
[6] European Commission, “AI Factories”, avaiable at https://digital-strategy.ec.europa.eu/en/policies/ai-factories, accessed on 23 February 2025.
[7] European Commission, “Coordinate Plan on Artificial Intelligence”, available at https://digital-strategy.ec.europa.eu/en/policies/plan-ai, accessed on 23 February 2025.
[8] Financial Conduct Authorit, “AI Update” (22 April 2024) available at https://www.fca.org.uk/publications/corporate-documents/artificial-intelligence-ai-update-further-governments-response-ai-white-paper, accessed on 24 February 2025. This document was in response to the Government’s publication of “A pro-innovation approach to AI Regulation: Government response” and “Implementing the UK’s AI Regulatory Principles: initial guidance for regulators”.