Dr Jeffrey Howard,[1] Dr Beatriz Kira[2] and Professor Marc Stears[3]—written evidence (LLM0049)

House of Lords Communications and Digital Select Committee inquiry: Large language models

Summary

1. LLM Opportunities and Risks: LLMs have enormous promise to enrich democratic education and discourse, promoting a plurality of perspectives and increasing information access. But they also carry significant risks. Unconstrained LLMs may generate harmful speech, including misinformation and hate speech. Such speech can endanger individuals, erode trust in democratic institutions, and undermine the overall health of the digital public sphere. Consequently, firms designing and releasing LLMs have ethical responsibilities to mitigate these risks.

2. Ethical Responsibilities and Self-Governance: Just as human speakers have ethical obligations to avoid causing harm through their communications, LLM firms have ethical responsibilities to design artificial speakers in ways that mitigate risks of harm. Meeting these responsibilities requires firms to reduce harmful content in training data, and to define and enforce clear rules for the content generated by their systems. LLM firms can further reduce the risk of harm posed by their models by improving public awareness about their limits in providing accurate outputs.

3. Better Incentives Through Regulation: The current purpose-specific approach to regulation outlined in the Government’s AI White Paper falls short in adequately addressing risks that arise regardless of purpose. To address this, we propose the adoption of a cross-sector regulatory approach that requires LLM firms to conduct risk assessments and develop proportionate mitigation strategies. A practical approach could involve granting Ofcom the statutory powers to regulate LLM chatbots under the future AI Act—as Ofcom will already regulate search engines under the Online Safety Act.

LLM Opportunities and Risks (Question 2)

4. Large Language Models (LLMs) hold significant potential for businesses and the broader economy. They also offer new opportunities to enrich democratic discourse. They can expand information access, accelerate knowledge dissemination, and (when curated well) promote a plurality of perspectives.[4] Yet this potential may be outweighed by the considerable risks that LLMs pose to public discourse, unless those risks are adequately mitigated. In this submission we focus on one specific risk: that LLM chatbots will generate speech that causes harm—to individuals and to democratic institutions. (These threats correspond to Risks to Safety and Risks to Societal Wellbeing in the Government’s AI White Paper.)[5]

5. Specifically, unconstrained LLMs risk producing many of the same varieties of harmful speech that have toxified social-media platforms. Unconstrained LLMs can produce misinformation: by repeating false content found in their training data; by “hallucinating” it; or by being maliciously prompted into generating material for disinformation campaigns.[6] Misinformation can harm individuals (for example, with medical misinformation); it can undermine the fairness of political processes (as with electoral misinformation); and it can undermine democracies’ information ecosystems by subverting norms of truth and trustworthiness. Unconstrained LLMs can also generate other forms of harmful speech with which social-media networks have grappled. For example, unconstrained LLMs, fed training data replete with prejudice and bigotry, are at risk of generating hate speech, reinforcing discriminatory attitudes and undermining democratic values of equal respect.[7] Without appropriate constraints, LLMs can also be abused to produce various forms of dangerous guidance—from bombmaking instructions, to tips on how to self-harm.[8]

6. It is uncontroversial that human speakers have ethical responsibilities not to cause unacceptable harm through their communications, either to individuals or to democratic institutions. We argue these same responsibilities should guide and constrain the design and regulation of artificial speakers such as chatbots. Chatbots are not moral agents themselves, of course, but their designers are. Firms must mitigate the risk that their models will generate speech that would be considered unacceptable if produced by a human. These obligations should be the basis of firms’ self-governance in this area, as well as any future legislation.

Ethical Responsibilities and Self-Governance (Question 5)

7. Like any company releasing products, firms producing LLMs have an ethical duty of care toward their consumers and to the broader public to reduce the risk that their products will cause harm. These responsibilities apply across the various stages of the AI value chain. (We use the term “LLM firms” broadly to refer to firms working across this space.)

8. The central way that LLM firms can reduce the risk of harm is to limit the harmful content that their foundation models produce in the first place. To live up to this responsibility, LLM firms must undertake both proactive and reactive measures. Proactively, LLM developers should work to exclude harmful content from their training data. More experimentally, firms might explore efforts to implement forms of “Constitutional AI” (as with Anthropic’s “Claude” chatbot), whereby models are explicitly programmed with clear constitutional values (although the effectiveness of this approach requires further study).[9]

9. Reactively, all LLM firms must establish and enforce standards for what counts as unacceptable content, filtering their models’ outputs accordingly. Such a system is analogous to the private governance systems that major social-media firms have devised and implemented to regulate the content on their platforms. Just as social-media platforms have progressively built more powerful and precise machine learning classifiers to catch harmful content, so too should LLMs. Moreover, given the impact these models may have on public discourse, firms have an obligation to be transparent about what their rules are and how they are enforcing them.[10]

10. LLM firms can further reduce the risk of harm posed by their models by enhancing public understanding of how their systems work and what their appropriate usages are. Scholars recognise that LLMs may not always track the truth; they draw on the vast training data of fallible internet text – a process that can often result in inaccurate responses (including “hallucinations” in which LLMs fabricate new falsehoods). There is an obvious danger that as chatbots enter wider use, users will assume that their responses are necessarily true. Firms can bolster public understanding through public education efforts, improved inclusion of disclaimers and caveats, and refinement of their chatbots to reflect greater humility and less confidence on reasonably contestable issues (as in, for example, a responsible Wikipedia article). Models can also be better trained to provide answers that do not confidently assert a single response but instead “map the debate” on a given question, which is especially important for philosophical, moral, and religious issues.[11]

11. Constraining LLMs to prevent harmful content production is the beginning—but not the end—of a theory of LLM firms’ ethical responsibilities. There is a strong argument for redesigning LLMs to actively promote democratic values, ensuring the inclusion of diverse perspectives and voices. Democracies thrive on a diversity of sources of intelligence, and chatbot responses should align with this pluralistic approach (rather than reinforcing a single worldview).[12] This mirrors the challenges for social-media platforms, where, even after determining which speech to disallow, the question remains of which perspectives or voices should be algorithmically amplified. A similar issue arises with generative AI.

Better Incentives Through Regulation (Questions 3, 4, 5)

12. The Government’s AI White Paper rightly emphasises an innovation-friendly approach to regulation. The potential benefits of generative AI for our economy and democracy are substantial. This explains why firms’ duty is to reduce the risk of harm, rather than to eliminate the risk of harm. A duty to eliminate all risks of harm could only be discharged by abandoning foundation models altogether. Such a stance is currently unduly pessimistic.

13. The AI White Paper is also correct that risks of LLMs (like all AI) can vary depending on their purpose.[13] However, the risks of harmful speech we have flagged here are purpose-insensitive, in that they do not depend on the goal for which the LLM is deployed. LLM outputs should never include hate speech or dangerous misinformation, regardless of whether they are being used as a search engine chatbot, customer-service chatbot, a medical adviser, or an email-writing assistant. The government’s position—that it “will not assign rules or risk levels to entire sectors or technologies”,[14] as set out in the AI White Paper—is too limited; LLM technology engaging with public audiences for any purpose carries risks of serious harm.

14. Mitigating these risks requires cross-sector regulatory action. The White Paper recognises that LLMs’ multipurpose character “means they are unlikely to be directly ‘caught’ within the remit of any single regulator”; yet it nevertheless concludes that it is currently “premature to take specific regulatory action in response to foundation models including LLMs.”[15] We believe this is a mistake in the context of harmful speech. The delay in demanding proactive content moderation for harmful speech on social-media platforms illustrates the perils of a “wait and see” approach. It was a serious error in that domain, and we should not repeat that mistake in the LLM space.

15. It is promising that firms such as OpenAI forbid certain uses of their LLMs, such as searching for self-harm guidance[16] (although this framing misleadingly places the burden on users, which is not always apt). These policies are the beginnings of a fully-fledged content moderation system. Yet much more work needs to be done. It is far less transparent how content moderation operates in the LLM space than in the social-media space. (There is nothing remotely akin to Meta’s Oversight Board.) Given that firms will not always have strong incentives to moderate content, or to explain to the public how they are doing so, democratic oversight of this process is essential.

16. Accordingly, we advocate for the establishment of risk-based regulation of LLMs, akin to that  originally proposed in earlier versions of the UK’s Online Safety Bill. This would require firms to undertake risk assessments for their LLM products, publish their risk evaluation and proportionate mitigation strategies, and be subject to oversight and evaluation by a regulatory body, which would be responsible for evaluating the adequacy and effectiveness of firms’ efforts. This oversight, when pursued in a proportionate and cooperative manner, need not hamper innovation. As with social media, content moderation will never be perfect[17]—false positives and false negatives are inevitable—but regulation can ensure LLMs firms have a coherent approach and are sticking to it.

17. In the UK, there is an opportunity to build upon the expertise of Ofcom, which is being strengthened to enforce the upcoming Online Safety Act. As currently designed, the Act will address user-to-user services and search engines, but LLM chatbot services (like ChatGPT) arguably fall outside its scope. To address this gap, the UK AI Act should introduce legal responsibilities for firms that design and release LLM chatbots, and also empower Ofcom to oversee firms’ compliance.

18. It is worth noting that even if LLM chatbots did fall within the scope of the Online Safety Act, the Act (as presently constituted) would do little to mitigate their dangers. That is because (with respect to content seen by adults) the Act focuses on speech that is currently illegal. But this is an unduly narrow focus for LLM regulation, for several reasons. Even if LLMs are producing text that could be criminal if communicated by a human, chatbots (and their designers) lack the requisite mens rea (mental element, e.g. intention) for criminal speech. Moreover, the harms wrought by LLMs to the information ecosystem are “legal but harmful”; even if humans should remain free to communicate misinformation (with some exceptions), the free-speech argument is comparatively weaker for chatbots. While overzealous regulation of LLMs could still trigger free-speech concerns due to its impact on audiences and users, that is compatible with our central point: LLM risk assessments should transcend a myopic focus on currently illegal speech.

19. Hence, we strongly urge the government to take immediate action and establish regulatory safeguards while conferring regulators with the statutory powers needed to supervise these evolving technologies. Specifically, empowering a regulator such as Ofcom with statutory authority to oversee the use of LLM chatbots employed across diverse contexts is essential.

About the UCL Policy Lab

The challenges of our disrupted world require genuinely transformative ideas. Led by Professor Marc Stears, the UCL Policy Lab brings together world-leading data-driven expertise from economics and political science with the lived experience and practical wisdom of governments, businesses and communities. Our collaborations aim to shape new debates and find new answers. Current programmes include Ensuring Sustainable Development; Challenging Inequalities and Protecting Rights; Promoting Democracy and Peace; and Rethinking Economic Policy & Decisionmaking.

About the Digital Speech Lab

Funded by a UKRI Future Leaders Fellowship awarded to Dr Jeffrey Howard at UCL, the Digital Speech Lab hosts a range of research projects on the proper governance of online communications. Its purpose is to identify the fundamental principles that should guide the private and public regulation of online speech, and to trace those principles’ concrete implications in the face of difficult dilemmas about how best to respect free speech while preventing harm. The research team synthesizes expertise in political and moral philosophy, the philosophy of language, law and regulation, political science, and computer science.

September 2023

6