21
Examination of witnesses
Owen Larter and Rob Sherman.
Q73 The Chair: This is the Communications and Digital Select Committee. We are continuing our inquiry into large language models. We have one panel of witnesses today. I will ask the witnesses to introduce themselves. Why do I not do that straight away? Mr Sherman, please introduce yourself and tell us which organisation you are here to represent.
Rob Sherman: Good afternoon, Chair and members of the committee. I am VP of Policy and Deputy Chief Privacy Officer at Meta Platforms. I have been at the company about 11 years and have worked on artificial intelligence for pretty much that whole time, whether that is using AI to help people see relevant content on our services or using it to combat harmful content. I am excited to be here and I appreciate the committee’s attention to this question of how we build artificial intelligence models responsibly.
The Chair: Thank you. Mr Larter.
Owen Larter: Good afternoon, Chair. Good afternoon, committee. I am the Director of Global Responsible AI Public Policy at Microsoft. This is a global role trying to share what we are learning from building our own internal responsible AI programme about how you identify and mitigate risks at a global level, working with countries around the world to develop and use AI responsibly.
The Chair: Thank you. I am very grateful to both of you for being here this afternoon and giving up your time. We have two big tech firms, Meta and Microsoft, before us today. Both have different business models in the context of AI and different approaches in terms of their commercial strategies, and perhaps different views as well on regulatory frameworks. We will be pleased to hear from you the differences on all of those things. We will cover how things are developing, open versus closed regulation, and copyright. I will go straight to Lord Hall to get us going.
Q74 Lord Hall of Birkenhead: Thanks very much for being here. I wonder whether you can start off by laying out the big picture as you see it. Try to help us distinguish between what is hype and what you think is reality, and the likely developments over the next few years. The question to you both—maybe, Mr Sherman, I can start with you—is how you see the next generation of generative AI models, tools and LLMs developing, on what time horizon, and what you foresee as the most game-changing capabilities over the next few years.
Rob Sherman: Thank you very much for the question. There is no denying that we are sitting at a important inflection point in the development of artificial intelligence. It is not that long ago that most of us had not heard the term “large language models”, and they have become a very important part of both the growth of technology and the way that our society will work.
When I look out over the next three to four years, I see three main areas where I think the technology is will grow.
The first is efficiency and performance, technologically making large language models more accessible and work better. When we released our large language model, LLaMA 1, not that long ago, it required very significant computing capacity to build and to use. Now it is possible to run a large language model on your computer. That has real benefits for small businesses, researchers, and people who want to use it but do not have access to a big data centre.
I would identify two other areas and we are already seeing progress in these areas. One is multilingual capability. Large language models have real promise for helping bring the world closer together so that people can communicate even if they do not speak the same language. You can interact with different systems—
Lord Hall of Birkenhead: In terms of translation?
Rob Sherman: In terms of translation, exactly. That is particularly valuable. A big area of work for us has been in what they call low-resource languages, ones that do not have large numbers of people with the ability to translate them. Helping to enable that is important.
Finally, there is what we call multimodality. Currently, if you use a large language model, if you use a chatbot on the internet, you will probably use it through typing with it and it will respond. Increasingly—and we have just put out some technology that allows this capability—we are enabling systems to understand visual inputs and translate them into words. For example, you could take a picture of something and ask a question about it. Helping computers to be able to do that is another important emerging capability.
Lord Hall of Birkenhead: I think your company has also made a point about this helping people who may not be feeling involved, minorities, disabled people or people who are feeling lonely and so on.
Rob Sherman: Yes, absolutely. One of the big areas that we have been investing in is looking at ways that artificial intelligence can be a force for inclusion. We have a series of what we call Five Pillars of Responsible AI, and fairness and inclusion is one of them. It is a core part of how we think about it.
Some examples of how this might happen: we talked about translation, but there is also the ability to use the multimodal capability that I talked about to understand the world around you. Let us say if you are experiencing blindness, you could have smart glasses that you could ask, “Tell me what is going on in this room”, and they would describe the space for you or be able to answer questions—those kinds of things, which give people access to the world around them.
The other thing that I should mention is that race bias and fairness is another big area of investment for us. When you build artificial intelligence models, it is important to make sure that they are fair and that they are not perpetuating societal biases. Therefore, a big area of investment for us is in both the ability to detect bias in machine learning models where it exists, and also to correct for that bias so that we are making the world a bit more fair and inclusive where we can.
Lord Hall of Birkenhead: Thank you very much. Mr Larter, the same question. How does Microsoft see it?
Owen Larter: We are very enthusiastic about the opportunity of AI. One of the things we have seen over the last year is people starting to use AI to enhance productivity and address major societal challenges. The conversational interfaces that large language models allow are democratising access to technology, writ large. I think you will see a lot of these trends continue. I think you will continue to see a growth in capabilities of models that will allow people to do all kinds of incredible things with increasingly capable frontier models. It will also bring some risks that we need to address, including by developing a regulatory architecture that maps to the technology architecture.
I would agree with Mr Sherman on the significance of multimodality. It will be fantastic for these models to be able to perform across various inputs and outputs. I think the accessibility applications are incredible. I also think about the educational domain, for example: being able to show an image to your phone and ask it, “How is this put together? Tell me more about what I am looking at”. That will be very powerful.
I think you will also see, over the next 12 months and beyond, a further maturation of AI safety. It is important to realise that the technology architecture that underpins large language models, these transformers, is still relatively new. It was created in 2017. We made a lot of progress quickly on the technology front; on the safety front, we need to continue making progress. I think about evaluations, for example. How do you come up with effective red teaming and systematic evaluations of highly capable models so that we can be confident that we are identifying risks and addressing them in a way that is satisfactory?
Lord Hall of Birkenhead: Thank you very much. I will ask you both this this question about the UK specifically. At one point we seemed to be arguing a lot about the UK being a place of great opportunity, and then we pulled back and the debate has become more about how we manage risks and so on. Could I ask you bluntly—you are here in London—what you think the great opportunities are for the UK in this world? I will start with you, Mr Sherman.
Rob Sherman: As you point out, we need to think about both the opportunities and how to manage the risks. The two go hand-in-hand. It is one reason I have been encouraged by the approach that the UK Government have been taking in thinking about artificial intelligence, because it is not one or the other. You cannot have the benefits unless you are thinking up front about managing the risks.
When I think about areas for opportunity, they include some of what we talked about earlier in terms of making this technology available not only to large organisations, not only to—[Interruption.]
The Chair: I am sorry about that. I wish I could control it, but it sounds like some passers-by on the river have decided to demonstrate, or it is a party ship. I am not quite sure which of the two.
Lord Hall of Birkenhead: Let us take it as approval of what you are saying.
Rob Sherman: There we go. Sadly, it is out of the control of all of us.
The ability to leverage this technology for small businesses is powerful. Think about a small business that is running an Instagram shop in their flat in London. They have the ability to reach a global audience but currently they have to spend a lot of time managing customer service, answering questions and doing these things that a larger business would hire a team to do. Large language models give them the possibility of using that to scale their business.
In terms of important things that I see the UK Government doing, just two weeks ago we had the safety summit in Bletchley Park, which I think was a seminal moment for the next phase of artificial intelligence. We need to be thinking not only about how business benefits, economic benefits and those sorts of things that I have talked about can happen, but what the risks are that we are worried about and what we need to do to build consensus on how to solve them. I think there is a real opportunity for the UK to build global consensus and global coherence around that. We are at this important moment where we have the ability to say, “Here is how we should be building AI safely”, and by getting global Governments together the UK has a real opportunity for a leadership role.
Lord Hall of Birkenhead: Innovation and risk develop side by side. Mr Larter, what are your thoughts about opportunities for the UK?
Owen Larter: The opportunity and the risk are very much connected and I think the UK is in a fantastic position to assist on both fronts. I do think the opportunity to use AI to accelerate productivity is very exciting.
We have what we call at Microsoft our copilots. These are not autopilots, these are copilots that help people do existing work more effectively. We have those tools across our Microsoft Office suite. You can use a PowerPoint copilot to help you design your slides, for example. We have a GitHub copilot, which is a tool used by software engineers to improve their productivity. The results are impressive. There have been 50% productivity increases already from software engineers using this.
We are already seeing the use of AI by the public sector in organisations such as the NHS, which we have been partnering with to do some work through our Microsoft Research Lab in Cambridge, using AI to do some of the basic preparatory work as part of radiotherapy treatment to get waiting lists down.
I think there is an incredible opportunity for the UK to use AI to supercharge its world-leading science sector. This is something that we are looking forward to leaning into further, including by Microsoft Research Cambridge producing new high-performing materials. That will have all kinds of cascading benefits right across society.
There are clearly risks, though; we are very clear-eyed about that, and those need to be addressed alongside the opportunities. An important thing to note about these risks is that, first, we believe that there should be a regulatory architecture that maps the technology architecture: application layer, model layer and infrastructure layer.
We also need to do this in a globally coherent way. AI is an international technology that is developed and used across borders, and it presents risks that transcend borders as well. The UK, as was evidenced by the safety summit, is an incredibly good place to try to advance that coherent conversation about global AI regulation. I think the Safety Institute is a important step forward. In the same way that the UK has led previously in advancing globally coherent financial services regulation, given the understanding and the skills and technology that you have in the UK, I think it can play a helpful role globally on AI regulation as well.
Lord Hall of Birkenhead: Thank you very much, and thanks for coping with the hooting.
The Chair: We will come on to regulation a bit more in a moment. I have a couple of quick supplementaries before we move on, going first to Lord Kamall and then to Lord Young.
Lord Kamall: Thank you very much. This is supposed to be amazing new technology and very exciting compared to older stuff, but if you think about the older programming there were always logics: if true, if false. One of the things I am worried about here is when we hear from witnesses about hallucination, inaccurate deduction or just making things up. What remedies can you put in place to avoid that, or will you just have lots of disclaimers saying not to take what we read as truth? People will take it as truth, whether you like it or not. I wonder if I can start with you, Mr Sherman, or whoever wants to come in.
Rob Sherman: You are absolutely right in saying that generative AI and large language models specifically are a different type of technology. They are probabilistic in the way that they operate. In our products that use generative AI we have tried to be clear about that fact, because it is important for people to understand what the technology can and cannot do.
That said, there are a number of steps that companies like ours—anyone who is deploying large language models—should be taking. That includes, first, modifying and taking care of the training data, which is essentially the data that goes into building the model, to make sure that we are doing things like removing private information and making sure that the information that goes into the model is safe and suitable. The second is training that we can do on the model itself, including something called reinforcement learning, which basically is telling the model how to behave and not behave.
Then the third piece, which is particularly critical, especially for a global platform like ours, is looking at the outputs of the model and adding in output filtering to understand whether the model is creating something that is problematic, but also—regardless of whether it is AI-generated or not—having the ability to detect harmful or problematic content on a platform and take it down.
Owen Larter: You are right, and these are important questions to be asking. It is important to remember that this is still very new technology that we are all learning about very quickly. I think things like hallucinations and performance will continue to improve, as they have improved already over the last 12 months, for example. I think that will be a big area of focus.
This point of transparency is absolutely fundamental. At Microsoft we have a series of transparency notes, as we call them. Across our Azure Cognitive Services, we publish information about the capabilities of the systems and also the limitations of the systems, use cases for which they are not suited and factors that will affect their use. That transparency will be important moving forward.
The final point to make here is about the importance of a risk-based approach for safeguards to the technology. It is obviously a different matter if you are using AI to serve someone up music recommendations as part of a content application, as opposed to if you are using a high-risk application to take a consequential decision about whether someone should get a loan or access to higher education. Where you have a high-risk use case you need a higher set of safeguards. You need to make sure that you are testing that system thoroughly so that it will perform appropriately in its chosen use case.
Lord Young of Norwood Green: I am not expecting a perfect answer to this question. You stressed efficiency and performance. Very briefly, what do you see as the impact on employment, both short-term and long-term?
Rob Sherman: It is important to be thinking about the role of employment. In some ways it is early to see the impact, but if you look back over the course of the past 20 or 30 years, we have seen lots of changes in the way that employment works. There are new jobs that technology has enabled that were not available before.
Artificial intelligence is an important leverage point for increasing productivity in lots of different jobs. I think it will make it easier for people to work. It will increase the ways that people can be creative and express themselves. There is certainly no end to the creative work that people will be doing and so I think there will continue to be opportunities for that, but the hope is that it will help people to be productive and handle more things in their daily work in an efficient way.
Owen Larter: There is a huge opportunity on productivity to use AI to advance highly productive, low-inflation growth now and into the future.
The way that our Microsoft copilots are already being used by customers is really instructive. I will talk again about the GitHub copilot, used by software engineers to increase the efficiency of their programming. It is essentially like an autocomplete for code, so that you can automate a lot of the repetitive tasks that you might do if you are a software engineer. The research on that is fascinating. First, it shows that people using it think that it is highly productive and useful. Secondly, they enjoy their job more. It frees them up to do the less automated, less repetitive tasks that are higher-impact and more satisfying. So I think that productivity approach will be helpful.
As with previous transformative technologies, AI will have an impact on the world of work for sure. An important piece of this will be making sure that people have the skills needed be able to use the technology. If you think about how much of a differentiator and a game changer this will be, we want everyone in society to make sure that they are able to use this AI to the best of their abilities in doing their work. Broadening the conversation out, we want to make sure that children right through the education system continue to have the skills needed to build this technology and then also use it once they leave school. And, of course, we need to continue to support people that are impacted by technology as they move from one job to another.
Lord Young of Norwood Green: Thank you.
The Chair: Thank you. I might have a supplementary of my own, but I will save mine and see whether we have any time at the end.
Q75 The Lord Bishop of Leeds: One of the examples you used earlier, Mr Sherman, was of a small business and how a small business might benefit. That opens up questions about viability, affordability and access. I would like to take us into questions about open and closed source models, which the committee has been very interested in with other witnesses. Do you believe that open or closed models are best placed to get the best balance between innovation and risk?
Rob Sherman: It is something that our company has been very focused on. We have been very supportive of open source, along with lots of other companies as well as researchers, academics and non-profits, as a viable and important component of the AI ecosystem.
There are a couple of reasons for that. The first, as you mentioned, is access for small businesses and small researchers. If it requires a large data centre and a lot of resources to be able to build one of these large language models and deploy it, very few companies in the world will have that opportunity. So giving that opportunity to the ecosystem, I think, is critical.
I should also say that it is not only altruistic. There are other benefits too, including to companies like ours. The first is that open-sourcing helps us improve the technology as a community and move faster in improving the technology. When the community is working together, rather than a small number of people at one company or a few companies, everybody benefits from that. We also see security improvements. Take the technology that we see across the internet, the core technology that allows us to be on the internet right now or to use a website, that is all open-source technology that is considered secure because it is open source.
Finally, there is a lot of value for open source in terms of transparency and accountability, allowing Governments, researchers, experts and other companies to understand how different models are working and their impact on our society.
All that said, we are not dogmatic about this idea of open source. Some of the discussion that both Mr Larter and I have alluded to about how to build AI responsibly includes thinking up front about situations where open source is not the right solution, and we have certainly made that decision as well.
Owen Larter: We are very enthusiastic about all of the innovation and experimentation that is going on across the AI ecosystem at the moment. We see a world in which we will have an enormous number of models—some big, some small, some optimised for certain things—now and into the future. Some of those will definitely be open source. Some of those will be made available in other ways. We are enthusiastic about open source at Microsoft. We contribute to the open-source ecosystem. We have our GitHub subsidiary, which is a repository of large amounts of open-source code. We also open source some of our own models as well.
I do think there is a conversation to be had about some of the trade-offs between openness, and safety and security. Particularly when we come to these highly capable frontier models—again, the middle layer of this technology stack that I mentioned—we need to take a risk-based approach there. When there is the next generation of models, which might have significant capabilities to do all kinds of good things but also may present serious risks, including of misuse, we need to think very carefully about whether it makes sense to open source those models or not.
The Lord Bishop of Leeds: Ian Hogarth, who was chair of the Frontier AI Taskforce, now the AI Safety Institute, said, “A closed system gives a Government more control in certain ways. It gives you something closer to an off switch ... which you do not necessarily get with an open system”. Would you care to comment on that?
Rob Sherman: It is certainly true that there are additional challenges with open source in the sense that you need to make sure that it is thoughtful before it gets released. That is why a lot of the discussions that happened at the AI safety summit will be important. What are the steps that we need to take to assess these models and make sure that they are safe to be put into the world?
I think it is important for us to be concrete about the risks that we are thinking about. Instead of a generalised risk of a model being out there and those kinds of things, which has been a lot of the discussion to date, it is important to go to the next step. Let us identify the specific kinds of risks that we are worried about. In a lot of cases, we have existing toolsets to assess and deal with those risks.
When we think about the kinds of things that the AI Safety Institute will be looking at, we are talking about some of the longer-term risks. People talk about sci-fi-style superintelligence. By most accounts we are pretty far away from that, but we obviously need to think about it. Then there are nearer-term questions along the lines that we have been talking about, around misinformation, safety harms and things like that.
In each case, what we should be doing—I think this is the important next step after the safety summit—is to think: what are the risks that we are thinking about, what are the tools that we have to assess whether those risks exist, and what are the things that we need to do in order to mitigate them? That might include not releasing or releasing in a more limited way.
The Lord Bishop of Leeds: Earlier you referred to the word “altruism” and things not being entirely altruistic. Given that the big tech companies like yours will benefit hugely economically and financially from whatever is developed, what is the primary driver for the developments that you are working on? Is it the benefit of the company or is it what you can call altruism, the good of society?
Rob Sherman: I think if we are doing it right, we can achieve both things that are helpful to the company and the goals that we have set, and also that are helpful to our society more broadly. At the highest level, we think about artificial intelligence as a tool that will help people connect and get closer together as a society and as a world. That includes things like being able to do business, being able to create and, as Mr Larter said, being able to be more productive.
From a pragmatic standpoint, having tools to be able to create is good for our platforms. People will use our services more if they have more ability to create content that they want to share with other people. Equally, there is a benefit to society when this technology moves more quickly and when we are able to work together to build artificial intelligence capabilities that we did not have.
One good example of this is, as I mentioned before when I talked about performance, LLaMA 1, our first large language model. It required very significant computing capacity to build, and by open sourcing it, it is now available to run on a laptop computer. That is not something we did, that is something that the community did, and that kind of thing is very powerful.
Then the final thing is security. We have done a lot of work on LLaMA 2, for example, to do red teaming and other adversarial testing. That is both work that we have done internally with experts to look at risks like privacy, safety, bioweapons and things like that, to test the model to see whether it raises those kinds of risks, but also work externally with people who we have not engaged, people who are part of the community and have an interest in this. Pressure testing it is a proven way, at this point, of making sure the technology is beneficial. In all of those ways we are improving technology for the world but it will also have benefits for us.
The Lord Bishop of Leeds: Very specifically for the UK, Mr Larter, do you think open or closed source models add to safety in its development in the UK and within the UK regulatory environment.
Owen Larter: It comes back to this risk-based approach again. There is a role for open-source models. There will be this rich model ecosystem that I mentioned.
When it comes to those highly capable frontier models in the next generation and beyond, which pose significant risks—someone could intentionally misuse them to perpetrate a cyberattack or more easily synthesise a bioweapon—we need to be very careful about open sourcing those models. We need to make sure that we have an appropriate regulatory framework all up for those highly capable frontier models—one that ensures that people who are developing them are doing so in a responsible way, identifying risks through testing before they launch them and having appropriate security protections around them. When it comes to those highly capable models, there is a question around open source for sure.
Baroness Harding of Winscombe: I would like to follow up, Mr Sherman, on your comments about the immediate near-term risks on misinformation. Particularly, I want to understand whether you think existing frontier models are making the risks of misinformation worse today, and if they are, what you are doing about it.
Rob Sherman: Thank you for the question. I would give two pieces in answer.
The first is on this concept of “frontier model”. One of the challenges that we are all facing right now is that we have articulated this concept of a frontier model, and nobody can agree on what it is or is not. One of the big areas of focus is to think about how we define thresholds for understanding what is creating additional levels of risk. I think that will be an important area of focus.
When it comes to misinformation, as you point out, it is not a new phenomenon—it certainly predates generative AI and also predates social media—but there is no denying that generative AI raises new and different risks when it comes to misinformation. First, our policies apply with equal force regardless of whether something uses artificial intelligence to create it or not, which means harmful content that violates our community standards will be taken down. We take aggressive action to do that.
Generative AI does give people who are engaged in sharing misinformation additional ways to scale the creation of the content, and that is something we are quite focused on. We have a transparency report that we put out periodically, and the next one, which I am happy to share with you once it is ready, will be talking about the work that we are doing on this risk. I think focusing on the content regardless of whether it is AI-generated or not is an important leverage point.
The last thing I will say is that we are also doing a lot of work, and I think this is cross-industry work, through an organisation called the Partnership on AI, which includes companies like ours but also Governments, non-profits and academics that are focused on the space. We are working together to build coherence around how we signal, both on the face of the content and technologically, that something is AI-generated when it is. For example, when you create an image using our image creation generative AI tool, it has a watermark on the face of the image that lets you know that it is AI-generated. We have just announced a new policy where we require political and issue advertisers that use generative AI to disclose that, and that is prominently disclosed.
We also just announced a new technology called Stable Signature, which is a research technology that we hope will become more of an industry standard, which allows you to include an invisible watermark in an image so that it is possible forensically to determine that an image came from generative AI. That is another piece. It does not solve the problem, because in a lot of cases purveyors of misinformation take genuine content and then share it in lots of ways with inappropriate or unhelpful context, but these multiple layers of protection are important for making sure that we are dealing with this threat, which is ongoing, and particularly focusing on the role that AI can play in both creating more risk and also combating it.
The Chair: I am sorry, were you going to add something to that?
Owen Larter: If I may.
The Chair: Just briefly, if I could ask you to be brief.
Owen Larter: Sure. It is important that AI is not used to fan the flames of misinformation. The provenance tooling that we have been talking about is very important. We use a tool internally based on the Coalition for Content Provenance and Authenticity standard, C2PA—it is a coalition that we are part of; we founded it alongside BBC, Adobe, and others—which allows you to indicate if a piece of audio or visual content has been generated by AI and allows you to provide information across the lifecycle of that piece of content if it is being manipulated.
We think this is important in the context of elections in particular—we have just made our election protection commitments a few days ago—so that the people who will exercise their democratic rights over the next 12 months and beyond are able to tell when content has been AI-generated.
The Chair: Thank you. Before we move on, one of you mentioned in passing unsafe models and being able to recall them. Is that something that is possible now? If a model emerges that is identified as being unsafe, can you do something to stop it being deployed further or recall it, like any product recall? How does that work?
Rob Sherman: It depends on what the technology is and how it is being used. Certainly, one thing that is quite important is to think about these things up front before they are released, as we have talked about. That is the primary forcing function that makes sure that we are doing this.
There are a number of other measures that we can take. For example, once a model is released there is a lot of work that what we call a “deployer” of the model has to do. There is not only one actor that is responsible for deploying this technology. When we released LLaMA, we put out what we call a Responsible Use Guide, which talks about the steps that a deployer of the technology can take to make sure that it is used safely. That includes things like what we call fine tuning, which is taking the model and making sure that it is used appropriately for particular purposes, and then also filtering on the outputs to make sure that when somebody is using it in an end capacity the model is being used responsibly and thoughtfully.
The other piece of this, which exists regardless of the technology, is thinking about acceptable use policies and measures that both private sector companies and Governments can take to make sure that technology is used in ways that comply with the law, and also the requirements of whatever licence the model is released under.
The Chair: I am conscious of time. Unless there is something really exciting that is different to what Mr Sherman has just said, can I move on?
Owen Larter: Please.
The Chair: We will stick with Baroness Harding, but she will move us on to a different topic.
Q76 Baroness Harding of Winscombe: Mr Larter, I will address this question to you. You referenced earlier that you think that there should be regulation of potentially future frontier models or large language models. I wonder if you could start by telling us what regulation you think should be put in place for these technologies as they emerge.
Owen Larter: We talk about the need for a regulatory architecture that maps to the technology architecture and these three layers of the technology stack.
At the top of the stack you have applications, things like Bing Chat, Microsoft Copilot, or a high-risk AI system being used to take a consequential decision. At that layer, we think that you should apply existing laws on a sector-by-sector basis and support existing sectoral regulators to do that.
Then at the model layer and the infrastructure layer, we think you need a licensing regime focused on those highly capable frontier models that I mentioned—those beyond the capabilities of today’s models that may pose some serious risks. We think that as part of that licensing regime you should require a few things. The model developer should demonstrate that they have very thoroughly tested the model prior to releasing it, red teamed it and addressed any risks that they have identified. We think it is important that there are security protections around those models so that people cannot steal them and then intentionally misuse them. We also think that information sharing is very important, continuing to share information with Governments and others about how this technology is working so that we can all learn and mitigate these risks together.
At the infrastructure layer, we think it is important that the operators of the AI supercomputing infrastructure on which these highly capable models are being trained and run are licensed to do that work. It is important that these supercomputer pieces of infrastructure have top-of-the-range security protections around them. We think that is an important part as well.
The final piece that I would mention is that it is very important to have this globally coherent approach to regulation. AI is an international technology. You will need to have the UK and others working around the world so that key elements of regulation are interoperable with each other.
The work that we have seen through the summit, setting up the UK AI Safety Institute and doing that in concert with the US AI Safety Institute, as well as what we have seen from various other countries around the world saying that they also want to set up their safety institutes and have a conversation together about these frontier risks, is very helpful. We think that we need to make further progress in that direction.
Baroness Harding of Winscombe: Thank you. Mr Sherman, do you agree that we should be regulating these models?
Rob Sherman: I broadly do agree with what Mr Larter said. I think the first point he made is that we are not operating in a blank space here. We have existing regulatory regimes that deal with many of the risks that we are concerned about. Certainly, here in the UK, the Information Commissioner’s Office particularly has been quite active in identifying both data protection and related issues, how we think about auditing AI to maintain consistency with data protection regimes and things like that. That is quite important.
I question a little bit the point that Mr Larter made around licensing. That works for large companies like Microsoft or Meta that can deal with regulatory regimes that require upfront approval. There is a lot of additional overhead that only larger companies will be able to manage. When we look at some of the other regimes that exist in the world, there are ways to do that require upfront assessments of the type that we are talking about without the requirement of licensing, which has the effect of limiting competition. However, broadly, this idea of applying different levels of protection to different levels of risk, and having very clear metrics and mechanisms for assessing risk, is critically important.
Baroness Harding of Winscombe: What does an upfront regime look like that is not a licensing regime?
Rob Sherman: We have seen a number of proposals for this around the world. We have looked at this and in the EU there are some proposals around this. It is saying that there are upfront assessments that companies are responsible for making. They have to make those assessments and if they hit a certain threshold they have requirements associated with them. There is not a requirement proactively to go to the Government and have the Government perform licensing requirements and licensing assessments, but there is an obligation to do and document that kind of assessment before you put a model into the field.
Baroness Harding of Winscombe: Okay. Again, just to check my understanding, could there be something between those two that is not unlike the accountancy audit requirements? Government does not sign off every company’s annual report and accounts but there is a very clear, standard way that companies have to demonstrate that their financial reports are accurate.
Rob Sherman: Yes, that is exactly right. The idea would be that there are very clear standards that we have agreed on, and I think this is one of the big next steps coming out of the safety summit. Let us agree on what the standards are and what is required. Companies that produce these models or that use them are required to meet those standards and document that they have done so, and then a regulator can come behind, look and see how they have done that and assess their work. However, there would not be a requirement to limit the number of companies so that only the big tech companies, for example, could do it.
Baroness Harding of Winscombe: That is very helpful. One final follow-up question if I may, Chair. It seems to me that one of the trickiest and most unique issues that these very large models bring in the regulatory space is where liability should lie if there is a problem, given what will end up being quite a complex supply chain. I would hazard a guess it will be more complex than the three layers that you set out, Mr Larter. How do you think we should think about liability through that supply chain? Should that all lie with the model developer, with the application developer, or should it be divided up?
Rob Sherman: As you point out, in some cases there is a quite complex supply chain of different actors that are involved in using this technology in different ways, and I think that each of them has a role to play.
When you are talking about the builder or provider of a foundation model, that requires a lot of work around the training data. What training data exists and how is the training data that goes into the model created? How do we make sure that the models are transparent about what their capabilities are and are not, so that people who are using them do not use them beyond their capabilities? I think there is a level of responsibility there for companies that create the models.
Once somebody takes that model, which you can think about as the language centre in your brain—it has the ability to process language or generate language but does not do any creative work on its own—and then puts in into a consumer product, let us say, that might be a totally different company that is making a decision to take that language processing capability and use it for a specific purpose. Just like we think about product liability today, thinking about: is that suitable for that purpose? Is it being provided in an appropriate way?
Then I think there is space for the end users of the technology to be responsible, too. Just like any technology, if I take it and use it in an irresponsible way through my judgment, that is something I would have responsibility for. So it is hard to say that only one actor would have responsibility. I think it is at every level of the chain.
Owen Larter: Can I layer on a couple of additional points? Yes, I do think you will need to have responsibility across the value chain. Transparency will be an important part of making progress here as well. It is important that the developer of the model, for example, is transparent about the capabilities and limitations of the model, and about ways in which it should and should not be used. I mentioned our transparency notes before. A big part of the reason that we put those notes together is so that downstream customers have a good understanding about how to use and how not to use that model.
Then, similarly, those further downstream who are developing AI systems based on the model and putting those systems into use in the real world will have to take responsibility for the deployment decisions they make about how and where to use that system as well. I would add that to the discussion.
Q77 Lord Foster of Bath: Thank you both very much indeed for coming. It has been fascinating.
We appreciate that you have to deal currently with a range of regulatory regimes, but in the UK text and data mining requires purchasing a copyright licence, with a number of exceptions, not least for non-commercial activity. However, if the purpose of ingesting text and data for training an LLM is to create a commercial product, it is pretty clear, is it not, that a copyright licence would be required? The focus is on intent. Bearing in mind that we cannot talk about any outstanding legal cases, can you explain what steps you are taking to ensure that LLMs that you have developed, in the case of Meta, or are using, in the case of Microsoft, comply with UK copyright and data protection legislation?
Rob Sherman: I should start by saying I am not an expert in copyright, certainly not an expert in UK copyright.
Lord Foster of Bath: Neither am I.
Rob Sherman: Fair enough. Obviously, complying with legal obligations when we build our technology is incredibly important and thinking about making sure that data is used fairly is an important priority.
We have thought about this by thinking about the training data that we put into place and making sure that we are using public data, data that is broadly available, and building models based on that, as well as talking with others in the industry about creating mechanisms for people who own websites or rightsholders to signal their intent when it comes to the content that they are sharing: for example, if they want it to be crawled or not. There are some standard ways on the internet that that can be communicated.
One of the big challenges in the space is that copyright law, for obvious reasons, was created well before the idea of a large language model existed. One of the challenges for Government and for those of us in the private sector is that it will take a decade or more for this to work through the court system and for courts to figure out how they intend for copyright law to apply in this case. I know that some other Governments are thinking about updating copyright law to clarify points such as those you are making.
Lord Foster of Bath: Just before we bring in Microsoft, from what you have said you seem to imply that you are pretty confident that Meta currently has not used any copyrightable data in the training of your LLMs. Is that correct?
Rob Sherman: I am not the right person to address the question of specific data, copyrightable or not. I know that our approach is to comply with laws wherever they apply.
Lord Foster of Bath: There is the issue of the interpretation of the law rather than complying with the law. I accept your point that court cases over time may resolve this, but do you accept the basic premise in Meta that if somebody holds the copyright of something and you make use of it for commercial purposes, Meta should pay for a licence to use that material, specifically in the case of training LLMs?
Rob Sherman: I cannot speak to the legal conclusions and what the legal obligations are. I would say that I think maintaining broad access to information on the internet, including for the use of innovation like this, is quite important. I do support giving rightsholders the ability to manage how their information is used. However, I am a little bit cautious about the idea of forcing companies that are building AI to enter into bespoke agreements with individual rightsholders, or to pay for content that does not have economic value for them. It is important to think about how we make sure that this technology can be enabled while also putting rightsholders in control of their information.
Lord Foster of Bath: What is Microsoft’s view?
Owen Larter: This is a really important conversation to have. I think this discussion around copyright will have a big impact on how and where AI is trained and used. There are a few things to think about as we move forward. First, it is important to appreciate what a large language model is. It is a large model trained on text data that is basically learning the associations between different ideas. It is not necessarily sucking anything up from underneath.
We need to make sure that we have copyright frameworks that provide protections to rightsholders. If AI is being used to create content that infringes on copyright, that needs to be addressed. We also need to have frameworks in place that allow us to build on knowledge and generate new knowledge and tools that people will be able to utilise.
This is one of a number of issues where it is interesting to take a step back and look at how different countries around the world are grappling with this issue. You can look at a jurisdiction like the EU or Japan recently clarifying that there is an exception within their law for text and data mining within the context of training an AI model. There is a longer list of countries that have that type of regime: the United States, Israel, Korea. That is one thing that we will need to balance as we move forward.
We support the recent recommendations from the Vallance report about clarifying UK copy law and that there is an exception there for that text and data mining, and note the recent letter from research organisations and others that supported that.
One other point I want to make is that this issue of clarity is very important, and so we at Microsoft are trying to bring clarity to this conversation as well. We recently announced our Microsoft Copilot copyright commitments. If a customer of ours is using a Microsoft Copilot in a way that generates content that is then subject to an action under copyright law, we will assume the responsibility for the potential legal risks around that case.
Lord Foster of Bath: There is a difference between that, which I accept—I welcome what you say you will do—and the actual creation of the model itself, which is created, in most cases, for a commercial purpose. The creation of the LLM is what we are debating, rather than how it is subsequently used for a variety of different commercial purposes. Would you not accept that if you are using copyright material in the creation of that LLM for profit, then where the data is covered by copyright, the owner should be reimbursed? You said you wanted clarity and it seems to me that is a pretty clear view.
Owen Larter: It is important to bring that clarity. I also think it is important to understand that you need to train these large language models on large datasets if you are going to get them to perform effectively, and if you are going to allow them to be safe and secure as well.
Lord Foster of Bath: Nobody is disputing the complexity of then abiding by the legislation as I described it. I am merely asking whether or not at Microsoft you share the view that I have expressed. If you are using copyrighted data and material for a commercial purpose, then, however difficult it is to resolve it—it might be by universal agreement; there is a whole variety of ways we could do it and I accept it is difficult—do you think that the creator should be reimbursed?
Owen Larter: Again, we can look to jurisdictions like the EU and Japan and the approach that they have taken with that exception. There are also some competition issues here around wanting to make sure that training of large models is available to everyone. If you go too far down a path where it is very hard to obtain data to train models, all of a sudden the ability to do so will be the preserve of only very, very large companies. That is something that we will need to guard against as well.
Lord Griffiths of Burry Port: May I just say—
The Chair: Sorry, I have a couple of supplementaries. It sounds like you want to ask one as well.
Lord Griffiths of Burry Port: I wanted to follow this.
The Chair: You had not signalled to me. I have some other colleagues who did. Let me go to Lord Kamall, then Lord Bishop, and then I will come back to you, Lord Griffiths. I would ask you all to be relatively brief.
Q78 Lord Kamall: I want to pick up on that last point, because I can see a possible scenario, however likely or unlikely. If you are coming up against copyright laws that require you to pay for training data and you decide not to pay, that means that you are now working on a smaller dataset, which means the effectiveness of your large language model weakens. Do you foresee that scenario or do you think there is nothing to worry about here?
Rob Sherman: Large language models, as you point out, need to be trained on massively large datasets in order to be effective. If we want to enable the creation of large language models, in order to be useful and to be in the range of the kind of technology that we are talking about today, this needs to be possible. There is a lot of work to do in thinking about how copyright law should apply to that situation, but I think it is quite important to enable that while also giving rightsholders the ability to control how their information is used.
Owen Larter: I think that is right. We will, of course, continue to abide by the law. I think the law needs to be clear. There is a danger that if there is not sufficient or appropriate quality data available, you will have problems in performance and also safety, security and trustworthiness issues. It is really important to have broad and representative datasets, including to make sure that your systems are performing effectively across dimensions like bias and discrimination.
The Chair: Lord Bishop and then Lord Griffiths.
The Lord Bishop of Leeds: You may go straight to Lord Griffiths.
The Chair: Lord Griffiths.
Q79 Lord Griffiths of Burry Port: I just got too excited; I am sorry. It was Don’s question and the succession of answers.
As an author myself, my attempts to provide quotations in books I write lead me to have all kinds of discussions with people who ask me to pay them for what is their copyright, and, of course, I have similar entitlements. Something of that debate is not reflected in the way that these models are being built. Ordinary citizens are subject to this all the time. I do take the point that you need to train these machines on the widest—I do take that point, I really do, and I would hope that all my books are being scratched or whatever it is you do, scraped.
Lord Lipsey: “Scratched” means taken out of circulation.
Lord Griffiths of Burry Port: You get my point. There is common decency involved as much as anything else. Have I answered my own question?
The Chair: Was there a question there, Lord Griffiths? No. You wanted to reinforce the point that had been made. I understand. Lord Bishop?
Q80 The Lord Bishop of Leeds: That was partly my question. Is it possible to differentiate between what is scraped in order to develop the technology and so on, which we all understand, and the responsibility to handle that appropriately and legally? Is it possible to distinguish between the two, or is it that you have to scrape everything in order to make the models work, irrespective of the provenance of particular material?
Rob Sherman: I do not think I would distinguish necessarily between the scraping and the use in the way that you have described, but I do agree that both are critically important.
One of the things that exists, and has existed for decades on the internet now, is a mechanism that is called robots.txt. It is a file that every website has that says, “Hey, I want this website to be crawled or scraped”, or, “I do not want it to”. There is a standardised, programmatic way that one computer can tell another whether scraping is okay or not. That is an example of something that already exists that we could use to address some of these questions. The person who owns the website could make a choice about what they want to have done with that website or not, but it does not require individualised assessments of how we should think about this particular piece of content, which just is not feasible at the scale we are talking about.
Q81 The Chair: I have a couple of questions that I had refrained from asking at the time. Just before we close, it would be helpful to come back to them.
When we were discussing open and closed models, I got the impression from the comments that both of you were making that you are advocating what I might describe as a mixed economy of open and closed sources, which seems like a shift from both of you but coming from different perspectives. I wondered if you wanted to say anything more to help us understand. Have you shifted and why? Let us start with Mr Sherman.
Rob Sherman: I do not think we have shifted. As I said, throughout the past 10 years of building artificial intelligence through our AI research lab, open source has been a common practice. It certainly is not the only way of releasing technology. We had some voice language generation capabilities that we built in our research lab that we decided not to open source because we did not think that was the right decision.
It is important to recognise that open and closed forms of release have different pros and cons to them. We have talked quite a bit about transparency today. Regardless of whether there is an open or closed release, making sure that the world understands what this technology is and how it is managed is very important.
That is why we have made a commitment to things like what we call system cards, which are disclosures of how our AI systems are working, and the transparency paper that accompanied LLaMA 2 when we released it, so that everyone can understand its capabilities. Those kinds of things are possible regardless of whether something is open or closed. I would recommend that we think about the release mechanisms along a continuum, rather than as a binary where something has to be one or the other.
Owen Larter: We have been supportive of open source for a long time, more supportive than sometimes people realise. Again, as I alluded to previously, I think there will be a real mix, to use your language, of different types of models being designed, developed and made available in different ways. It is difficult to overstate the amount of innovation and experimentation that is currently happening in the AI ecosystem. A lot of that is open; a lot of that is using a different approach. That is all good and something we would encourage.
I would just bring us back to the point that when it comes to highly capable models that may pose a significant risk, that is where we need to have a serious conversation. It may likely not be appropriate to open source, and we certainly need to make sure we have good, robust safeguards around those highly capable frontier models in general.
Q82 The Chair: My final question is, in a way, a segue from that. When you were answering questions about the opportunities for the UK in terms of this technology, you seemed to be talking quite a lot about what I might describe as deployer opportunities. You are right, there are clearly lots of opportunities in that regard, but I did not get the impression that you thought there was much opportunity here in the UK for developing AI. Have I misunderstood? Have you concluded that we are not going to be developing a large language model or another form of foundation AI here in the UK, or that we do not have the capability? It was striking to me that that was not an area that you seemed to focus on. Mr Sherman?
Rob Sherman: I absolutely think there is development opportunity in the UK. We have a very large engineering office here in London and are very committed to the UK as a market. Whether we go—
The Chair: Not necessarily Meta or Microsoft, but perhaps competition to either of you.
Rob Sherman: Sure. lot of it goes to the questions that we have been discussing around whether the UK can create a regulatory regime that removes some of the barriers to being able to build this technology, but ensures it is done safely. The UK has taken some prudent steps along those lines, both in creating leadership for global accountability and global consensus on how to do this and also thinking about the regulatory regime.
We were just talking about copyright and that is a good example. Thinking about a practical way of moving forward and creating certainty for companies—whether it is Microsoft and Meta or others—that want to build this technology to know that there is a mechanism to do it and that they will not be stuck in litigation is an example of something that would be really valuable.
Owen Larter: There is a huge amount of opportunity in the UK on both the development side and the deployment side, and a huge amount of innovation. We are very excited to play a role as part of that with our Microsoft Cambridge research facility. Also, on the deployment side, there is an enormous amount of opportunity there.
One thing I would draw attention to is the recently announced executive order from the Biden Administration in the US. It is really interesting that it is both a framework for addressing the risk of highly capable frontier models but also a strategy around how to accelerate the utilisation of AI right across society—and particularly across the Federal Government—to advance efficiencies. They had pieces of the strategy in there around bringing talent to the US and investing further in research in the US, and they had strategies for different agencies across the Federal Government to identify where it might be helpful to deploy generative AI to advance efficiency. That type of integrated strategy could be very helpful, but for sure I think the UK is right at the head of the game in terms of being able to play a role innovating on the development side and the deployment side as well.
The final point I would make is about putting in place a sensible regulatory framework, both here in the UK and, again, at a global level through the work of the Safety Institute, the partnership with the US, and continuing to work through vehicles like the G7 to develop this globally coherent set of safeguards so that we can realise all of the benefits of this innovation responsibly. There is an enormous amount of opportunity here.
Lord Griffiths of Burry Port: Forgive me. I will just say, if I may, that the fourth Oral Question, which is about to be asked, is about the executive order precisely. Your timing was perfect. It is a pity we could not put it on and listen to what was happening.
The Chair: We will be very interested to hear what the Minister has to say in response to whatever question he has had down there. I did not realise that was happening today. There you go. We are multitasking everywhere in the House of Lords on AI matters.
Thank you both very much for your time and for being with us this afternoon. We are very grateful to you. Thank you as well for coping with the noise from the river. I am glad that in the end it was not just the window closing that managed to cope with that. Thank you.
To those who are tuning in to us, I should say that next week we will be hearing from what I call the research lab businesses, OpenAI, Google DeepMind and Aleph Alpha. We will be continuing with a different perspective on this next week. Thank you very much indeed.
