Financial Services Regulation Committee
Uncorrected oral evidence: Regulatory sandboxes in financial services
Wednesday 18 November 2026
11.15 am
Watch the meeting
Members present: Lord Hollick (The Chair); Baroness Bowles of Berkhamsted; Lord Davies of Brixton; Baroness Donaghy; Lord Eatwell; Lord Griffiths of Fforestfach; Lord Hill of Oareford; Lord Lilley; Lord Sharkey; Lord Smith of Kelvin; Lord Vaux of Harrowden.
In the absence of Baroness Noakes, Lord Hollick was called to the Chair.
Evidence Session No. 7 Heard in Public Questions 75 - 86
Witnesses
Benjamin Cave, Chief Product Officer, Aveni; Desmond McNamara, Chief Risk Officer, Zilch.
USE OF THE TRANSCRIPT
Benjamin Cave and Desmond McNamara.
Desmond McNamara: Thank you very much for the invite. I work for a company called Zilch, which was founded in 2018 and is a direct‑to‑consumer payments and credit business. We entered the regulatory sandbox in April 2019.
At the time there were lots of businesses being built that were providing merchant-subsidised credit—you would often hear it called BNPL—but those transactions being merchant-subsidised were outside of the regulatory scope. They worked under the exemption rule. You then had credit cards, which were in effect a regulatory product, so consumers had the protection of various consumer protection laws in regulated credit, but people were charged interest.
Our aim was to show that we could provide merchant-subsidised transactions within a regulated product. We wanted to apply for a consumer credit licence. We looked at whether we should apply directly through what I will call the traditional route. The sandbox programme was around. Working with our advisers, we felt that the sandbox programme was a more attractive method for us to engage in a dialogue and have a representative within the innovation area who might be able to provide us some feedback on our application, for what was really an innovative way of using regulated credit.
We exited the sandbox about nine months to a year later with a consumer credit licence. Since then, we have also applied for some other licences and gone through the traditional route.
Benjamin Cave: Hello, members of the committee, and thank you very much for having us along today. I am the chief product officer of Aveni. At Aveni, we are a provider of specialist AI solutions and tools to the financial services industry. We have been established for about seven years. We have customers and investors primarily in the wealth management space as well as in wider banking, including the likes of Lloyds and Nationwide.
We are veterans of two sandboxes; we still have sand between our toes. We entered our first sandbox, the digital sandbox, in 2023, as part of our training process for FinLLM, which is the UK’s first large language model specifically aligned to UK financial services regulation. We entered the supercharged sandbox towards the end of 2025 to test our real-time conduct guardrails and approval processes for AI agents handling consumer financial interactions.
Our aim in both cases was to engage directly with the regulator when it comes to these specialist compliance tools, and to get a connection to the forefront of what the regulator is thinking and its approach to regulating what is a very fast-moving technology. In generative AI in the first instance and then agentic AI in the second instance, our goal was to seek direct engagement with the regulator.
We had a good experience in both sandboxes. They put on a fantastic showcase. There were some areas where we would have liked additional support and where we think that improvements could be made in the future, but I will happily talk through those as we go through our questions.
The Chair: So for both of you it lived up to your expectations.
Benjamin Cave: From our perspective, it lived up to some of our expectations, such as the regulatory exposure. There were, however, some areas where we felt the sandbox constructs that we were exposed to—which is a different set of sandboxes from yours—could have been improved. It was primarily around access to data, not open-source public data, which is what we largely found inside the sandbox, but specific data developed with industry and the regulator, which could have helped us test those things faster.
There could have been more guidance and engagement from FCA experts throughout the programme. We felt like the FCA had a relatively hands-off approach to the programme, which maybe suited earlier-stage businesses that were just setting out on the development of their products, and for which things such as access to compute were very attractive, but we were looking for deep and ongoing engagement with the regulator and guidance about how we shaped our solutions. Those were areas where we feel it could have been improved.
Desmond McNamara: We have been very positive about our time in the regulatory sandbox. We have also supported other sandbox programmes. One of your previous presenters talked about the role of mentors to sandbox programmes. We actually provide staff to be mentors to other sandbox programmes from a commercial point of view, so they can have feedback as to how they could learn and develop their products and think more generally. We think it is a very positive environment.
Benjamin Cave: From our perspective, it certainly indirectly improved our products. If I talk specifically for 20 seconds, just to give you some context, our primary area of focus has been the ongoing safety and conduct of AIs when they are talking to the consumer, and being able to provide safeguards in the form of AI guardrails and testing environments, where financial services institutions can appropriately and adequately test the conduct adherence of those AI solutions against realistic consumers before they expose the consumer to potential harm.
In participating in the supercharged sandbox in particular, if I talk about that experience, we found that it was very validating. A lot of the businesses—and indeed the FCA—were very, very positive about the need for this approach. It provided us with a lot of confirmation.
Direct improvements to the product were more limited. I would say being unable to access the specialist datasets or regulator‑approved datasets that we were after limited the amount of direct exposure, but it did directly tee up our live tests with one of our banking customers and supported the exposure of our product to the wider industry.
Lord Sharkey: Are you saying you did not actually get access to all the data you wanted?
Benjamin Cave: No, we certainly did not. The data we found in the sandbox was largely public open-source data, which, as a relatively established business in the space, we already mostly had access to. What we would wish for in future is access to industry-developed datasets. If I take the reference point of Project MindForge, which the Singaporean regulator has been developing for the last two or three years, that is a direct collaboration between the regulator and industry to develop datasets and evaluations, which AI companies can then use as approved gold-level industry data. That was missing from our experience of the sandbox.
Desmond McNamara: For us, there was definitely a huge impact for consumers, because we were able to launch a consumer product in 2020, which was a regulated product. The rules on deferred payment credit, which came into existence last year, have taken nearly six or seven years to get to the point where consumers are able to have the benefits of consumer regulation. You could say that we were seven years ahead of where the regulations needed to get to.
Benjamin Cave: In our experience, whether you call it a sandbox or an incubator is largely a matter of taste. What it provides to the regulator—and this is a very clear benefit—is early visibility into the direction of travel for industry and providers. If I look at the different environments, the live testing environment is for more mature financial services providers. It gives the regulator early access to the industry’s thinking.
In terms of it being an incubator, the two-way nature of that relationship could be improved in future ones through some more intensive exposure of us as industry providers to regulatory experts in the FCA. We are in a very fast-moving field with AI. We understand the outcomes-based position of the FCA and its desire not to endorse things; we would never ask for that, but what would be helpful is a more sustained engagement of FCA experts with the nature of the solutions, so that providers can get a better early sense of whether their direction of travel is correct and in line with the regulation, or whether something needs to change. That would make it more into what I would recognise as an incubator construct.
Desmond McNamara: Generally, as a fintech we believe that the idea that there are multiple methods for firms to come and test their products with the FCA makes complete sense. We have a very clear understanding of what each of the different sandboxes does. I accept that maybe it is a little opaque outside of financial services, but the ideas that there are different pathways offered to firms and that firms can choose to pick the pathway that suits them the most makes complete sense to us.
I accept the challenge that firms will always turn around and say, “We would love there to be more data, or more relevant data”, but firms often do not want to share that data, because it is proprietary data and is data they are trying to use to build products for themselves.
The different sandboxes, as they are all labelled, do very different things. It is very clear to most people in the industry what they are all trying to achieve, but I accept that it may not give a firm everything it would always have wanted, because firms, by their nature, sometimes have to go and build their own products and services.
Baroness Bowles of Berkhamsted: Looking at the data point, because it has been referenced that you sometimes get access to datasets from the FCA, are you saying that they are not as good as some of the proprietary ones that there might be? Why could those datasets not be released more generally, so that people who are not in the sandbox could use them as well for testing? Is there something special about those datasets that you have to keep confidential?
Benjamin Cave: For us, it is less about access to a firm’s proprietary data. We would certainly accept that those are all part of that, but as a regulator, if we look at the role of regulators in the age of AI, being able to provide some level of gold labelling or interpretation of what is and is not acceptable over the top of even synthetic data would be extremely helpful to industry, because it provides you with guidance on the bounds of what is and is not likely to be accepted by the regulator when those products make their way through to market.
Again, if I come back to the example of Singapore, there has been an active focus there to provide regulator-annotated datasets to all of industry, and thereby to issue a much clearer level of guidance on what will and will not be an acceptable direction of travel. What that prevents, if that were possible, is providers of all shapes and sizes going off down blind alleys that later prove to be unacceptable to the regulator. Providing synthetic data that is non-sensitive to the businesses that participate in it would be tremendously beneficial to the whole industry, not just to one particular participant.
Baroness Bowles of Berkhamsted: We are not getting that at the moment; it is only if you are in a sandbox that you are lucky enough to get that here.
Benjamin Cave: Certainly from our perspective in the supercharged sandbox, the data that we found there was largely open-source data, which would be relevant only to a very nascent business just starting out.
Lord Hill of Oareford: Just to be clear, it is not that you are getting the data you want only in the sandbox; it is that you are not getting the data you want in the sandbox.
Benjamin Cave: That is correct.
Baroness Bowles of Berkhamsted: Yes, so there are two issues going on here. One is that you are not getting enough data in the sandbox, and then in the sandbox you are getting data that would actually be useful to the wider industry, and that is what happens in Singapore. Is that fair?
Benjamin Cave: With Project MindForge, which was the example we had in mind in Singapore, a lot of the largest players in the industry such as banks and credit unions work together with the regulator to develop non‑sensitive synthetic datasets, which could then be exposed to all the participants of sandboxes. A similar approach here would be beneficial to the industry as a whole, and to the UK’s position as a financial services leader.
Desmond McNamara: The challenge somewhat is that we do provide all our data. All the firms in the UK provide all their credit data to bureaus, and that is an entire business model where bureaus then sell that data back to firms, making that data visible and accessible for firms to go and buy and connect to today. The FCA is providing a synthetic view on them, but a lot of that data is out there for you to go and buy and use if you wanted to. We do not have a particular use case, because we have not used that digital sandbox, but the FCA is not the sole provider of data that is needed to be able to develop business cases.
Benjamin Cave: As a participant in the sandbox it is hard to establish whether the regulator’s approach changed. We have certainly fed into the AI good practice guidance, which was published last month. That was a welcome clarification from the regulator. Whether that directly connected to the sandbox, though, as opposed to the call for expressions of interest, is hard to ascertain from the outside.
The regulator is a key beneficiary, and that is as it should be. When you talk about the very fast-moving aspects of the field such as AI compliance and governance, it is right that the regulator is exposed to what firms are doing.
For us, and for our partners and clients, as beneficiaries, what would be desirable if we were thinking about improvements to the construct of sandbox in the future are more ongoing working groups and participation with regulatory experts. So much of aligning a principles-based regulatory regime with the very strict, rule‑based interpretations that AI primarily favours is about getting guidance from those regulatory experts on an ongoing basis for that product development.
That is not a problem unique to us as a provider of services to the industry. Industry players themselves regularly seek that ongoing clarity of how the regulator would view or interpret specific things. If that needs to wait for a live testing environment, that is sometimes a year or two of development in potentially the wrong direction.
Desmond McNamara: I would certainly say that we benefited from going through the sandbox programme at the start. We did not enter it solely so that the FCA could benefit. What we saw was that, when the deferred payment credit rules came around to be discussed, the process and journeys that we had built were used as quite a good example of how digital lending could be done in a regulated format. We met with the Treasury. There were FCA attendees there who had seen our business model and had seen our journeys, so they could definitely see what was possible from that activity.
It is also a responsibility on firms to have ongoing relationships and to build upon those. We attend as a mentor because we feel that there is something to give back. That builds our relationship with the FCA. It cannot just be a one-time, “I need something right now to test or get a licence”; it is an ongoing relationship. We have supervision teams who come and see us regularly. We have built a much broader relationship with the FCA. Does that help it think about other things going forward? I hope it will. It has benefited us as a firm and I hope it has also benefited the FCA in thinking about how to deliver innovative products in new ways.
Benjamin Cave: That is certainly part of what the FCA is grappling with, not just through the AI good practice guidance, but also with Mills as a review process and the clarifications or directions of travel that were issued there, which we also participated in feeding into.
From our perspective, we see the FCA maintain a very clear stance on outcome-based regulation, the senior managers regime, and the responsibility of senior managers within that, in essence boiling down to, “Whatever your AI solutions are, they devolve to the responsibility of the humans who are responsible for them”.
Our primary area of study with the FCA, if you want to put it that way, has been how you could ever satisfy a human that a very fast-moving AI system is remaining in alignment or inside of its regulatory perimeter. That remains a problem that needs to be solved by the industry and we continue to seek more engagement with the regulator so that it can issue more clarification around where that perimeter sits.
Lord Lilley: Mr McNamara, you mentioned at the beginning that someone in your position knows pretty well what the different sandboxes are and what they are for. Could you give us a quick note of what they are, not necessarily now? This has puzzled the committee for some while.
Desmond McNamara: I will not go through all of them, because there are several, but we can look at the most recent one, which is the supercharged sandbox. We are a regulated firm. We are governed by the senior managers regime and consumer duty. We have to generate good outcomes for customers. We are responsible for the journeys that we deliver. We are responsible for the onboarding process that we have for those customers, how they use our product and how they interact with us if they get into financial difficulty, for example.
We can apply—and do apply—AI models to that journey and that process, but we are doing that to have better outcomes, not worse outcomes. The framework that the FCA already has is that we have responsibility as a regulated firm to be able to implement AI models in a responsible way. We take that very seriously. We use models and test models, and there are very clear owners and managers of models.
We did not need to go through the supercharged sandbox to test that, but there are lots of firms that I have seen going through the supercharged sandbox that may not be regulated themselves today, do not have access to data, and may want to either sell an AI solution to a regulated firm or become a regulated firm. What they are seeing in that is the challenge of how you build models, control models and manage them.
There are fewer regulated firms inside the supercharged sandbox because we already have that responsibility and that is our day job. It is a service that the FCA provides to firms that want to come and test some of those new, innovative products to maybe sell to regulated firms, or to regulated firms that have never used them and want to have a testing environment.
Different sandboxes have unique deliverables. We understand that. That is why we are a mentor into that programme but we do not need to be an applicant into that programme.
Lord Lilley: The last panel referred to the lack of a cohort experience, which raised in my mind the idea that perhaps half a dozen different firms would come into a sandbox with similar or related products that they are developing. It might be useful for them to exchange ideas, but it might also be difficult, I would have thought, if they are competitors.
Desmond McNamara: That is why the usage of mentors is a great technique. We attended the kick-off meeting for the most recent cohort and were there to talk to all the cohort members. Any of them could come and ask us any question they wanted. It was almost easier for them to come and ask someone who had been through a programme, rather than maybe someone who was currently in their programme.
They also all tend to have very different business models. We found that the cohort members who had business models where they were trying to deliver solutions into our type of company would want to come and engage more actively with us. There is an element where, if there were more mentors from a broader range of firms, that might be helpful, but that environment is there.
Benjamin Cave: Not to stem from that, but to offer a different perspective as a provider into those firms, we would most certainly have found it helpful had there been a theme and a cohort of businesses that were alike in two ways. Number one, they were grappling with the same issue, but, number two, they were of roughly the same stage in their maturity and development.
We found with our cohort in the supercharged sandbox that many of the firms were pre product. They were extremely nascent. They might have had an idea or just be starting out on their journey. Some of the benefits offered by the sandbox to firms at that stage are tremendous, such as access to free compute and access to datasets, which otherwise would have been difficult or time‑consuming to lay their hands on.
For us as a more mature firm going through that process, direct partnerships with technology providers provide us with comparable or higher levels of free compute. We have already talked extensively about access to data. We maybe had a different set of concerns and priorities coming into the sandbox.
Grouping things in cohorts with a more common theme and a more common stage would allow you to centralise and concentrate the types of mentors that are available. You could have exposed people to more of the same type of mentor, which would ultimately get firms further ahead in their participation in the sandbox, and would also potentially allow the FCA to lay its hands on those with regulatory expertise in something more closely aligned to the theme. We found quite a few experts in second‑line practitioners, but not a lot of those who were expert in the first line about how this was applied.
Stronger themes and cohorts that are more closely grouped would benefit the firms going through it, as well as the mentors and the FCA, which get exposure to more of the problem solving at that stage.
Desmond McNamara: We went through a cohort where there were very few people doing what we did. While it would have been nice to have had the idea that there were other people we could talk to, I would much rather have got on with it and gone through, rather than wait for six months or a year until there were enough people to discuss the element.
The challenge with innovation is that you generally do not tend to get 10 people who think the same way as you all at the same time. I know you are not saying that directly, but it is a challenge. It would clearly be useful if there were a few more like-minded people facing exactly the same challenge you had, so you could discuss it with them, but the nature of innovation—
Lord Lilley: Would you not be in danger of giving them your secrets?
Desmond McNamara: Exactly, so would you really share that information with them anyway? If your product is so innovative, there is probably not someone doing what you are talking about. It is a problem that will never be solved. It is an objective that one cannot aim for.
Benjamin Cave: What becomes clearer and clearer through those two different answers, which are not in contradiction with each other, is the difference in perspective between suppliers to the industry and regulated providers of products. What we are saying here is that they benefit from different constructs in their sandboxes. Those journeys look rather different for those who are trying to gauge the FCA’s posture on something so that they can supply the industry, and those who are themselves suppliers of solutions to the end consumer.
Benjamin Cave: At one level that is a very expansive question. Where the regulator should be commended is that the FCA has always been a leader, and continues to be a leader, in terms of that focus on the outcome for the consumer, and the focus on the principles rather than the didactic rules that govern these things. In the age of agentic AI, where it is so tempting to issue hard and fast guidance, a focus on the outcome for the consumer is the right one to maintain.
That does, however, create quite a lot of ambiguity in the minds of industry as to how the FCA will treat or view a particular application of this. Will your model risk governance on the models you use be sufficient in the eyes of the FCA. If there is an incident or a redress, will the FCA look unfavourably on the senior manager for not having sought more deeply for assurances on these things, or will it consider the assurances that it was provided with sufficient?
These are the areas where more ongoing guidance and engagement with the FCA’s experts is very beneficial for industry, because those problems being sanity-checked or surfaced earlier in sandboxes and other forms of working engagement allow the industry a better guidance.
I will speak finally about the symptom of this. We have seen an awful lot of firms pilot the use of agentic AI directed to the consumer or behind the scenes. We have seen very few of those firms, including larger financial services institutions that we work with, make their way to production. The biggest obstacle to seeing more of the benefit of AI across financial services is a nervousness or an understandable risk from the second line and from those chief risk officers about how they will be viewed by the FCA if something did go wrong. More guidance and more engagement with the industry could only be beneficial in providing that level of confidence to second line and third line, and providing the industry with more opportunities to launch and get value from the solutions that it has developed.
Desmond McNamara: I am the chief risk officer for the firm, and I feel that the rules are more than sufficient already. Too-prescriptive rules will limit innovation. The outcomes-based approach is the right approach. There is always a benefit of having some thematic learnings that will then get shared back across the industry on a regular basis, but it is generally difficult to expect more very prescriptive rules to be written on a product or a service that has not even been built yet.
That thematic element is the right approach right now. Maybe in 10 years’ time they will become more prescriptive rules, but there are lots of rules on model governance today that are already in place. We have applied those rules for model governance when using models since the Fed’s SR 11-7 and have brought those into agentic AI. From our point of view, it is already there.
Benjamin Cave: From what we have seen in the use cases, I totally agree with you; it is not that more prescriptive rules are required, but it is a general nervousness about the way that those things will be treated. An easy parallel occurs here with self-driving cars. We have had the technology for self-driving cars for quite some time. What we do not have is the regulatory construct to say who is attributably at fault if the consumer suffers harm. For example, are we saying that the individual in charge of these agentic AI solutions will always be at fault? Is the harm foreseeable, or is the harm not foreseeable?
We are certainly not asking for more prescriptive rules, but more engagement with the regulator and more clarification of its thinking on these issues can only help the industry.
I am interested in more engagement. Let us assume somebody has done a common or garden sandbox. They then have the option of going into a supercharged sandbox. Is that the same as early and high-growth oversight, or is that different?
You also have the possibility of joining a provisional licencing regime. I wonder, therefore, how you see this and what your participation in it has been. Basically, if the FCA wants to engage with you more in an area that is changing at a tremendous pace, it needs to hire people with more skills in the business, who will be, by definition, more expensive. Is that the ultimate logic of where you are going?
Benjamin Cave: I am certainly not qualified to opine on some of those provisional licensing regimes. Desmond, you would have much more knowledge.
Desmond McNamara: I will cover those.
Benjamin Cave: What I can say from observation and from working with a number of clients in this space is that there is a gulf. You have accurately described the gulf. Those who are experts in the regulation are rarely the people building and maintaining the models. Those who are experts in the models are rarely the experts in the interpretation or enforcement of the compliance and regulation. The two of those things becoming closer together is in everyone’s interest.
I will not speak for Zilch or anyone else, but a pattern we have seen is that agents and AI solutions are developed functionally, with very little reference to regulatory oversight, model risk governance or foreseeable consumer harms, and then they are subjected to those at a later stage. That is a very high-friction process, because designs are not built with that at the front of the process.
In terms of the regulator, our greatest need is for the regulator to be able to grapple with new problems with the industry, not specifically with us as an individual provider, but through the sandbox constructs and other forums of engagement, in order to think through those more nuanced examples: “When is a harm foreseeable in the construct of an AI system? When should I have satisfied myself with more evidence?”
At the moment it feels like the principles are crystal clear, but in the interpretation of those principles into this technology and this industry there are many, many grey areas currently being worked through very differently by many different providers and firms. The FCA’s sustained engagement with that over repeated working groups, as opposed to individually or at a point in time through a showcase, would be of great benefit.
Desmond McNamara: To answer some of the particulars you had, the early and high-growth group is a dedicated supervision team that engages with a firm once it has become regulated and has been awarded a licence. It is there to help firms through the first 12 to 18 months of being regulated, because there are quite a few returns you have to make on a regular basis. It is just allowing firms to have a point of contact into a general group that can help if there is a new return and you do not totally understand how to fill that in.
The FCA does not only learn via sandbox programmes; it runs a lot of thematic reviews. Our door is always open, and the FCA opens it quite regularly to come and ask us questions. It asks for our input. Lots of consultations are sent out. There are consultations on credit reporting changes. There are consultations on CONC changes that are going on.
That process of getting feedback from firms is there. I accept it is incredibly complicated, because we have a legislation framework in consumer credit that primarily has run since the 1970s. We are trying to evolve this environment for the modern world. There is a lot of two-way communication between the FCA and firms. Have I answered your question?
Lord Griffiths of Fforestfach: Absolutely, yes. In terms of higher quality, I do not want to in any way question the ability of people who presently work with the FCA, but given the technological changes that are taking place, when you are playing in that division, you need to be with people who are at the cutting edge. They are expensive, but the logic of what you are saying seems to be—if I was advising those in the FCA—that they really need to be on their toes about appointing the right people, because the pace of change is just so great.
Desmond McNamara: We are all learning how to manage AI models and developing those controls, in the same way that within a firm we have a very clear view that there would be a model owner, but there is a second‑line review and challenge of that. We are upgrading our own skillsets to do that. We need to be able to explain how these models work in a simple way to a knowledgeable, sensible person. We do not need to be able to explain it to a PhD mathematician. It is that thematic approach: “Show me whether it created harm. Show me where it was beneficial”. You can sometimes have quite simple questions and boil it back down.
If I can just give you one tangible example, firms can have a call centre where they engage with customers, but most call centres close at 7 pm. If a customer has a question at 9 pm, or is maybe feeling very vulnerable at 10 pm or 11 pm, and they want to get an answer, surely it would be better to give them a very good answer using an AI agent that has certain controls and limitations than ask them to call back at 9 am the following day.
There are automatically things where we can say, “We can insert a model into that environment under controlled rules and under controlled usage”, which is already making consumers’ lives better. We can sometimes boil this down into tangible activities, rather than get lost in the detail of how the underlying model works.
Benjamin Cave: That goes to an extent. We know whereof we speak, because we have one of Europe’s largest AI labs working on financial-service-specific models, but it relies on the premise that governance and controls like this, once established, remain. We know from practical examples that models can drift hour to hour. A model that gives a perfectly good answer to a consumer at 9 am might be unwittingly engendering harm for that consumer at 10 am.
How you monitor those things in real time and what level of evidence and satisfaction a senior manager needs to properly feel accountable for the ongoing performance of that model is something that the regulator will have to wrestle with. Traditional, set-in-stone governance policies and methods, even as they are applied through model risk to generative AI, are increasingly inadequate to agentic AI.
Desmond McNamara: My responsibility today as a chief risk officer is to ensure we have the controls in place to do that. You are right; that is the challenge. A firm doing it well will have those controls in place and do that. As firms, it is our responsibility to deliver good outcomes to consumers. It is not the FCA’s responsibility. The FCA has set the broad framework for the rules and regulations for firms to operate within, but we have very clear guidance: we have to have good outcomes for our customers when we use these models.
Benjamin Cave: As far as that goes, that is certainly the case. Specifically to the point you were making in terms of the FCA’s staff and thinking about this in the future, as those grey areas become larger between the very clear focus on good outcomes and the way in which firms choose to interpret and implement that, and the decisions those firms make, the FCA will need to continue thinking—and upgrading its thinking—about where it should issue guidance into those grey areas, and where it should allow them to develop a variety of solutions.
What I do not understand is the extent to which all this work promotes innovation. The FCA originally said that its aim for the regulatory sandbox—which is the one most of us have in mind when we talk about sandboxes—was to promote innovation and competition. I have seen all the evidence that shows that they facilitate innovation. What I am not so clear on is that they actually promote innovation.
In practice, I am sure there is a big overlap between the two, because if it is facilitated you are more likely to do it, but what is the counterfactual if we had not had sandboxes? We have a counterfactual in the States, where they do not have sandboxes and we are told there is a lot of innovation going on there.
Benjamin Cave: That comes down to the central question, at least for us as a provider: why seek engagement with the regulator? You eloquently made the point, and I hope we have as well. Firms are innovating anyway. Firms will get on with innovation. Our primary priority in our commercial lives is to seek the clients who need that innovation and to find the opportunities there.
The reason for turning our face towards the regulator is to get as close as is reasonable—not all the way—to some form of endorsement, positive steer, or at least a lack of negative alignment with the regulatory regime as it stands now and potentially thinking into the future. We do that because as suppliers to regulated industries we do not have the same certification regime that gives us an FCA gold stamp of our processes. We cannot get an FCA endorsement, and, to be very clear, we would not want one. We understand that the regulator must be assiduously neutral here. Our reason for engaging with the FCA and the innovation it can help to promote is at least in being able to say to us, with some confidence, “Having seen your product and the way it works, it is not out of line with the way that we think about the regulation”.
How could that be improved? That could be improved by having a pathway, whether that is a sandbox, sustained working group engagement or some other construct, for a firm to be able to show the FCA more of what it is working on and at least get some guidance that nothing in the way that it is working on this problem is out of line with how the regulator would treat this. From our experience, that is what most of the firms in the supercharged sandbox were after, and what none of us quite received from our participation.
Desmond McNamara: It is difficult for the FCA to create innovation. It is for firms to create and be innovative. What the FCA has created is an environment we feel we can operate within. These multiple pathways, while complex to understand, create different opportunities for firms to go down. The idea that one size fits all is not what we want. There are multiple approaches for doing different things.
Fintech firms are particularly very good at testing and learning. “How do I show a particular page to a customer? Do they understand that?” They are good at getting feedback from them and evolving. The idea that the regulator would also sometimes chop and change some of these pathways—“Are they being taken up? If they are not being taken up, let’s adjust it”—is consistent with the way that we operate. I am sure the regulator has lots of things where it thinks, “Wow, I could improve this and that”.
The UK is an area where there is positive support for innovation. The FCA plays its part in that. We certainly feel that we can innovate our products here; that is why we are here.
Lord Davies of Brixton: Is this more about better innovation, rather than more innovation? Let us call it the pathway to innovation.
Desmond McNamara: It is for firms to innovate. If you need a licence, you need data, you need to test some technology or you want to look into, as a previous speaker had, crypto and tokenisation, those are all areas where firms can approach the FCA and engage in some way, shape or form to deliver something. The innovation is on us. The requirement is on us to deliver products and services that people want to buy and use.
Benjamin Cave: Your example throws up an area where we are not seeing the engagement that we need to, which is where our thinking gets ahead of the FCA. I will give you a concrete example from Aveni. We think about how we bring a real-time guardrail, which is a check on an AI model’s response to a customer, down to where it can be applied at the message level. Before every message gets sent back to a Zilch customer, it would go through a guardrail check faster than the blink of an eye, where you could say, “The conduct of this agent has been safe in this message”, and pass that on. When we talk about that to the regulator and we ask, “Will this be an evidencing expectation?” our thinking has advanced ahead. That is as it should be with innovation.
At that point, though, we need a sustained conversation with the regulator to say, “Imagine you had a world in which this was the case. How might regulation be applied to it?” It is about getting more illumination and clarification of how regulation might be applied into those grey areas in the future. That feels absent today, because, with the stand-off that this is principles-based and the firms will innovate, those firms—and we work with a lot of wealth managers—still operate under the uncertainty of what the FCA might get around to tomorrow and decide to have a thematic review of, have a particular look at or change the guidance on.
Being able to engage early with regulators on the way that they are thinking about certain emerging problem statements is a benefit to everyone. Whether a sandbox, a pathway or some other term is the right way to do that is something we could explore more.
Benjamin Cave: Our costs are fairly minimal. As you would expect from a provider, we already had the technology, so our costs were really just the people and engagement cost, and the opportunity cost, I suppose, of dedicating those resources to engagement with the FCA, as opposed to with our clients or on innovation of our products.
The costs were certainly not burdensome. The FCA did a good job of setting up a streamlined application process. We found it very straightforward to participate in the sessions. The showcase was slick and well organised at the end, and gave us good exposure to industry. The costs of engagement were minimal, at least for a provider such as us.
Desmond McNamara: I am not aware of there being any specific costs allocated to a firm for applying to or being in any of the sandbox programmes. There is a cost of that engagement, but that was a cost we wanted to take, because we wanted to have that engagement. To us, there was no additional marginal cost. In fact, we feel it was time and money well spent, because it allowed us to explain our product and services versus the more traditional route. We have separately used the traditional route, where we had a more traditional product change that we wanted to do and therefore did not need to have that additional engagement. It is for firms to tailor as to what works best.
Desmond McNamara: Anyone can apply to go on to these mentor programmes. It is then up to the individuals who are on the cohort as to whether they wish to reach out to any of the mentors. We describe our background and experience. It is then up to people to see whether it is relevant. Several of them have found it to be relevant, so we proactively engage. We do that in FCA programmes, but we have also done that with founders forums and scale-up groups. If we have been through an activity, we try to give back. I hope our learnings and feedback are useful to those people and, if not, they will tell us.
Lord Hill of Oareford: Was the genesis of it the FCA thinking it would be helpful to have a group of experienced people who have been through it to help other people go through the process, or did it come from the industry saying, “Could we have such a group”?
Desmond McNamara: I am not sure where the genesis of it was, but the pitch is that it is a service that the FCA provides, which is dependent upon mentors choosing to join and upon the cohort members choosing to find those mentors useful. If there is no match, there is no harm done, as such. There is no activity done.
It is a useful element for the financial service ecosystem to learn from itself and to challenge itself. We learn as much from just the energy and enthusiasm of some new innovative companies. Sometimes it can tick in our mind, “That’s not relevant to us in that way, but I can see how we could take that to our product in a different way”. We learn as much as we give.
Lord Hill of Oareford: Mr Cave, this is a version of Lord Griffiths’s question. You said earlier you thought it would be helpful if you had more expert AI specialist regulatory input. Are you not always going to know more about developments in AI than the regulator possibly can? How does that work? How can the regulator keep up? How can it in practice give you that extra help that you might want?
Benjamin Cave: It is more about understanding what the future regulatory posture would be. I do not want to skew this to all firms. You have heard from Zilch that this is not relevant to a lot of providers, but as a supplier to the industry the main thing we really want from the FCA is to understand at least the direction of travel in forthcoming regulatory posture.
Lord Hill of Oareford: This is the Catch-22. I do not know how you can reasonably expect the FCA to give you guidance, since it does not really know, perhaps. This is my question. Does the FCA understand what is going on in AI? How can you expect it to work out what the future regulatory direction of travel is going to be? That is not really in its gift, because it is politicians who might set the legislative framework or whatever it is.
Benjamin Cave: In terms of changes to the regulation, I fully accept that. That lies outside the FCA’s gift, but how a principles-based regime and a focus on outcomes are interpreted in the context of developments in AI is something the regulator can provide one side of. The other side is industry.
I suppose my answer to that question would be that ongoing working groups between the FCA and multiple providers, not just a single provider—I recognise the difficulty of spending that much time—would allow for a wider conversation that would, on the FCA side, raise its level of awareness or understanding about the directions of innovation or the ways in which the boundary is being pushed by AI providers.
On our side as providers, it would provide us with an understanding of how the FCA regulatory experts, for what of a better word, respond to those. Do they raise concerns about how that would be interpreted within the senior managers regime? Do they raise concerns about specific kinds of foreseeable harm that need to be controlled for? That lets both sides learn from each other in an ongoing relationship. As opposed to a point in time within a sandbox where you are potentially having one short conversation, it is an ongoing and sustained engagement that benefits both sides.
I wanted to pursue Lord Hill’s issue about mentoring. The Zilch submission mentioned the phrase “effective regulatory engagement”, and you yourself in your evidence today have used the phrase “multiple methods to test a product”. When you applied in 2019, the word “sandbox” would have probably been ultra-new and exciting, and everybody would have said, “Ooh, a sandbox, gosh”. I am beginning to question whether it is a useful word now, and whether phrases such as “engagement”, “multiple methods to test a product”, “mentoring”, would be more useful, first for the outside world to understand, and secondly for newer firms coming in to understand precisely what the process is. Do you think we are just making a lot of fuss about nothing? Are you quite happy to keep using the word “sandbox” even though it does not mean anything any more?
Benjamin Cave: On a personal level—and this truly is a personal response—I am not a lover of the word “sandbox”, even from back in the early days of the regulatory sandboxes. It creates a slightly infantile impression of playing around with something. In this day and age, what we are talking about is a slightly more sober engagement exercise. From a nomenclature perspective, it is probably due for a clarification.
In the light of how the focus of sandboxes has evolved between what was true in 2019—well before the advent of generative AI—and now, it should be something that confers more of a sense of sustained engagement, not simply opening up an area where people can experiment, but experimenting with them or working towards something. I again take the reference point of Singapore, where they are called projects. If we had something that conferred a sense of ongoing engagement and trying to solve a problem together with industry, that would perhaps be a useful rebrand.
Desmond McNamara: I totally see your point. We are at a unique point in time, where there is going to be unprecedented change in financial services over the next 10 years. We will see 30 to 40 years’ worth of change in the next 10 years, so we need an environment that allows the UK to benefit from that. That is what I call the sandbox. You are creating a broad set of different environments to adjust regulatory interaction to have the best outcomes where the UK can take advantage of that. People can innovate new products. They can launch new services. The UK can show how we can do this in a very controlled and regulated manner.
I do not know what I need to call it going forward, but that is the aim and the objective. That is actually the aim and the objective of many of these activities. I am less concerned about the naming of them. That is what UK Inc is trying to do, and that is what the FCA has been responding to. The scale-up unit and suchlike are a response from the industry, saying there may be a need to help firms a little bit more in that scale-up phase. Do we know exactly what it is yet? The answer is no, so there is a first test cohort of this. It may turn out to be a terrible idea and not to be needed, in which case it should end.
The idea of putting some element of support in there to see whether it is useful to achieve that objective makes complete sense to me. I am less worried about the naming, but I agree the naming probably makes it complex for those who are not directly in there and maybe prevents some people getting involved, but regulated products are complex by their nature.
Benjamin Cave: If I can build on top of that, the danger with sandboxes is always that you set them up, you announce them, they have grand PR, firms join, they are left largely alone, and then you close them down with a showcase.
I can contrast that to something else the FCA is doing, which is the live testing regime. Providers enter with serious consumer‑facing AI solutions and get sustained engagement from the regulator and sustained feedback on their level of regulatory alignment of their solutions. That feels much more like a direction of travel. That is an AI testing regime where the regulator stays in and provides real, meaningful feedback to the firms. The sandboxes, albeit aimed at earlier-stage businesses, could learn a lot from the seriousness of the engagement there. Likewise, as I have said in previous answers, the FCA could learn a lot from engaging with the industry at earlier stages, not simply its largest players.
Baroness Donaghy: This is a subjective question. Do you think the FCA is better at its concierge service than it was six or seven years ago?
Desmond McNamara: The idea that a sandbox programme is a single point in time and an endpoint is the wrong mindset. Sandboxes on their own are a means to an end to do a particular thing, to have a discussion about a particular licence, or to test some data or AI solution. In the background there is also a responsibility on firms to have an ongoing relationship with the FCA and to build that engagement and that feedback. That is done via mentors, thematic reviews and a number of different things.
How that all evolves most efficiently is a valid question, but you can see that the building blocks of it there. It is for others to think about whether it is going to change, but I can see how it is a requirement on firms to carry on that engagement post being in a sandbox. We have taken our responsibility very seriously from that point of view.
Benjamin Cave: From my perspective, the slight misalignment is the nature of the benefits. If I look at the supercharged sandbox in particular, it was driven on the primary benefit that you would get access to computing provided by Nvidia and datasets provided by the FCA. That is the wrong focus, in my view, as I have made clear in previous answers. For me, the primary value of engaging with the regulator is to get exposure to the regulator. Almost all these firms, even at the earliest stages, are awash in free compute.
I can see why having the FCA go together with large providers of infrastructure to these firms benefits infrastructure providers such as Nvidia or Microsoft. I can see why it benefits the FCA to announce those partnerships. It terms of what the firms are seeking, it focuses on the wrong part of the problem and gives freebies at the expense of maybe longer-term engagement with the regulator, which is what is so hard to come by for these firms at this early stage. It is not the compute; it is the airtime and sustained attention with the regulator.
Desmond McNamara: We feel that we have been able to achieve that. What is it that we have done right that others have not been able to achieve?
Benjamin Cave: You are a provider of regulated products. You have a certification regime. For those of us who are guessing where the FCA will go next, or innovating a generation ahead in regulation, it becomes a lot harder to get that when the FCA has not issued certification regimes or clarification.
Desmond McNamara: That is a good way to separate it. For regulated firms, that does transition. Maybe for service providers to regulated firms it is a little bit outside of the box. Then again, back to your earlier point, the challenge is that it is a thematic-based set of criteria, so it is for the firms to describe what their appetite is, where they are going and what they think is right.
Benjamin Cave: Our lived experience of working with the largest wealth managers and banks says that that is not a widely shared view: that it is all on the firms and we have all the clarity we need to interpret this properly. That is not something our largest banks or wealth managers share.
Benjamin Cave: There are two levels of worry to that. First, as was alluded to earlier, there is a valid justified concern about the presence of mentors and industry providers in these, which I have heard from a number of firms. It is less of a concern for us, but I know it is a concern for many of those we went through the sandbox with. The presence of larger institutions looking in on their innovations and their work and then, as you put it earlier, transposing that work to their own context does provide a threat. It does provide an incentive not to showcase all your IP in these environments, because those showcases are exposed to a lot of the largest providers, which might choose to go it alone rather than purchase from you.
Between firms there, our experience was that most were at the early stages of forming their innovation or did not have significant IP to present. From our own perspective, we did not publish most of our IP that we walked into the sandbox with as a result of the sandbox there. We published a lot of the testing results of that.
Firms have protections available to them, certainly through the construct. I did not feel it was anti-competitive for us to get together, but there is less of a pathway than some of the other trade associations provide, for example, to being able to get direct meetings and exposure with industry, and providers being able to speak to the largest financial services firms as a result of their participation.
Desmond McNamara: I have been to several of the showcase events. Generally, for things such as the digital sandbox and the supercharged sandbox, when you get to the end of the cohort experience, each of the attendees presents. Those are generally quite open forum invites where anyone can attend. It is up to those firms to decide how much they want to say, but, if they are building a service that they want to sell, it is a good test for them to try to sell to a series of people in there.
We do not build our entire architecture end to end. We work with lots of partners, so we might bump into a new partner at one of those events and think, “Wow, that is a really interesting idea. We don’t need to build that ourselves, but we see how that could fit into our environment”. That is an acceptable outcome. The showcases are there to show off their innovation. If you want to find out more, you can then catch up with the individual firm. It is not the case that there is any restricted information. They are very open environments. “Do you want to go? Do you have the mindset to go and see? How do you engage with them?” I found them quite positive.
Benjamin Cave: They are an excellent industry platform. The FCA does a good job of producing those events and of putting together a good set of attendees for the firms to present to. That side of it is most certainly really positive.
The Chair: Thank you very much for prompting a very useful and helpful discussion. That concludes the session.