Dr Joseph Stubbersfield and Dr Alberto Acerbi—written evidence (LLM0024)

House of Lords Communications and Digital Select Committee inquiry: Large language models

Human-like content biases in Large Language Models

Dr Joseph M. Stubbersfield is a lecturer in Psychology at the University of Winchester. In his research, he uses experimental methods to examine how psychological biases influence the dissemination and communication of information including misinformation and conspiracy theories.

Dr Alberto Acerbi is an Assistant Professor in the Department of Sociology and Social Research at the University of Trento, and member of the Centre for Computational Social Science and Human Dynamics. In his research he uses computational models and quantitative analysis of large-scale cultural data to examine contemporary cultural phenomena. He is author of the book Cultural evolution in the digital age.

This evidence is submitted in response to the government’s call, so that the Communications and Digital Committee is aware of the implications of human-like content biases in LLM produced texts. Specifically, it responds to:

Question 3: ‘How adequately does the AI White Paper (alongside other Government policy) deal with large language models? Is a tailored regulatory approach needed?’

Particularly question a) ‘What are the implications of open-source models proliferating?’

1. LLMs are currently used, or their use has been proposed, in journalism,[1] copywriting,[2] academia[3] and other writing tasks[4] and the proliferation of open-source models will only increase the dissemination of LLM produced text in wider culture. 
2. The risks of LLM texts reproducing human biases or stereotyping, particularly gender- or race-based prejudices have been broadly acknowledged[5],[6] and are mentioned within the AI White Paper.

3. However, research in social learning and cultural evolution demonstrates several psychological biases which influence the dissemination of information in culture. In addition to stereotype-consistent information,[7] human memory and social transmission demonstrates biases towards content which is negative,[8] threat-related,[9] about social relationships,[10] and counterintuitive (i.e., counter to intuitive expectations about physics, biology and psychology).[11] These ‘content biases’ have been demonstrated using both experimental methods and through analyses of cultural data, showing that these biases are present in humans, and influence the content of wider culture.[12]

4. Further, recent research shows that LLMs demonstrate biases for content analogous to humans. Across five experiments, replicating social transmission experiments in humans, text produced by the LLM ChatGPT demonstrated content-based biases for gender stereotype consistency, negative information, threat-related information, social information, and information which is biologically counterintuitive.[13]

5. The issue of LLM biases for these contents is not currently addressed in the white paper. As these biases in LLM output align with biases in human psychology, they may be more difficult to recognise objectively and may have consequential downstream effects. In particular, through processes of emotional contagion in digital media,[14] negativity and threat bias in LLM generated material could contribute to broader negativity and overestimation of threats in culture. Given that negativity and threat biases likely play a role in the dissemination of misinformation,[15] and conspiracy theories online,[16] they could also contribute to the role of LLMs in the spread of misinformation.

6. When considering proportionate measures for bias detection, mitigation and monitoring, regulators should have an understanding of how LLMs reflect human biases, including subtle biases such as towards negativity and threat-related content, and the implications of such biases on the dissemination of information (and misinformation) in wider culture.

7. Summary

1. Large Language Models (LLMs) are proposed as having potential to be used or are already being used in a wide range of writing tasks, including journalism, copywriting, academia.

2. Research in Cultural Evolution and Social learning demonstrates that humans are biased to attend to, remember, and transmit some types of information content over others.

3. In our evidence, we articulate that LLMs show biases analogous to humans for content that is gender-stereotype consistent, negative, threat-related, about social relationships, and biologically counterintuitive.

4. The presence of these biases in LLM output may be difficult to detect, and has potential to magnify human tendencies for appealing, but not necessarily informative, or valuable, information content and could contribute to broader negativity and overestimation of threats in culture, and the appeal of online misinformation.

September 2023

3