Written evidence from by Dr Sophie Nightingale (Lancaster University) [MIS0066]
About the respondent
I am a senior lecturer based in the Psychology Department at Lancaster University. My research focuses on the harmful uses of technology and examines human and computational approaches for distinguishing between real and AI-generated content. I was awarded a UKRI Future Leaders Fellowship in 2024 which focuses on protecting ordinary people from the harms of deepfake technology—specifically, this work investigates ways to better protect women and girls from AI-generated sexual digital forgeries. The evidence I am providing is based on my academic knowledge and understanding of the issue and is given in a personal capacity.
I am happy to attend in person to present this evidence and answer questions from the committee.
Questions addressed:
- How is content on social media and online, such as violent pornography, contributing to sexist and misogynistic attitudes and behaviour and violence against women and girls both online and offline?
- What role are social media companies and online platforms playing in the promotion of misogynistic content?
- What strategies and interventions are needed to address misogyny among young people?
1. Executive summary
1.1. Advances in generative Artificial Intelligence (AI) have led to an increasing prevalence of sexual digital forgeries, specifically ‘deepfake pornography’ and ‘nudify’ apps/websites.
1.2. These sexual digital forgeries are a violence against women and girls (VAWG) issue—98% of AI-generated videos online are non-consensual sexual imagery of which 99% target females; and ‘nudify’ tools have been designed to work specifically on the female body.
1.3. Sexual digital forgeries can perpetuate and promote rape culture and sexual violence against women and girls.
1.4. This digital manifestation of misogyny works to intimidate and silence women and girls online.
1.5. Legislation is being updated but there are many challenges with enforcement, including differences in legality across jurisdictions, lack of police resources and specialised training, difficulties regulating tech platforms/companies, and the sheer volume of such content.
1.6. Existing research shows people have limited ability to accurately distinguish between real and AI-generated facial images. The perceptual realism of sexual digital forgeries is largely unknown due to the ethical considerations of such research; however, the harms are real regardless of content provenance.
1.7. In my lab, we are conducting research to support better protection of ordinary people from the harms that sexual digital forgeries present, and we are designing a verification system to detect such content.
1.8. Targeted academic research, improved education, pressure on tech platforms/companies, and sensible legislation that can be flexed to keep up with evolving tech must be implemented to slow the wildfire spread of sexual digital forgeries and the associated promotion of online misogyny.
2. Sexual digital forgeries: definition, prevalence, and impact
2.1. Generative AI allows for the digital creation or alteration of content to show people saying or doing things that they’ve never done. A common use of generative AI is to create non-consensual sexual imagery, referred to in this document as ‘sexual digital forgeries’.
2.2. One common form of sexual digital forgery is ‘deepfake pornography’, where AI is used to swap someone’s face into existing sexually explicit content or to generate novel sexually explicit content. Another type of sexual digital forgery involves using ‘nudify’ apps or websites to remove clothing in images of adults/children without their consent (https://www.bbc.co.uk/news/articles/cgr58dlnne5o; https://www.bbc.co.uk/news/articles/cr78pd7p42ro).
2.3. Although the term ‘deepfake pornography’ is often used, both words in this context are problematic: The term ‘deepfake’ was coined in 2017 by a Reddit moderator who created a subreddit for users to share celebrity ‘deepfake pornography’; this origin places alternative terms such as generative AI or synthetic media as preferable. Further, the term ‘pornography’ implies the content is consensual between adults and therefore other terms such as sexual digital forgeries or non-consensual synthetic intimate imagery are preferred (McGlynn & Toparlak, 2025).
2.4. In 2019, Deeptrace found that 96% of online deepfake video content was of ‘nonconsensual deepfake pornography’ with 99% of this content featuring women and girls (https://regmedia.co.uk/2019/10/08/deepfake_report.pdf). In 2023, Security Hero revealed that 98% of deepfake videos online were ‘deepfake pornography’ and 99% targeted women (https://www.securityhero.io/state-of-deepfakes/).
2.5. Further highlighting the gendered nature of digital sexual forgeries is the finding that ‘nudify’ apps or websites have not been designed and developed to work on men (https://www.nytimes.com/2024/03/23/opinion/deepfake-sex-videos.html).
2.6. As well as general pornographic sites, sexual digital forgeries are shared on specialised deepfake porn sites and circulate in the manosphere promoting and normalising "rape culture and sexual violence to women" (http://bbc.co.uk/news/articles/cewgxd5yewjo/). In addition to the dehumanising content typically depicted in these forgeries, discussion space comments perpetuate and fuel misogynistic and degrading views of women, in turn encouraging more extreme content (Hall, 2011; Shor & Seida, 2019) and likely leading to misogyny and sexual violence in the physical world.
2.7. These discussion spaces also offer a place for commissioning of not-safe-for-work (NSFW) deepfake media, offering financial incentive for creating sexual digital forgeries.
2.8. Sexual digital forgeries are increasingly prevalent due to the democratisation of the software required to create them. For example, a 2023 report found a 550% increase in the number of deepfake videos available online since 2019, and a 464% increase in ‘deepfake pornography’, specifically, between 2022 and 2023 (https://www.securityhero.io/state-of-deepfakes/#key-findings).
2.9. The growing prevalence of sexual digital forgeries paired with the lack of societal response and lack of consequence for perpetrators is normalising and trivialising this crime. The sheer commonality of this content is making it seem acceptable—in turn, this digital manifestation of misogyny works to intimidate and silence women and girls online.
2.10. There is a misconception that ‘fake’ content shared and viewed in the online ecosystem does not cause harm to victims. The harms are real and there is evidence suggesting that the psychological trauma caused by virtual harm is akin to that caused by physical harm (https://www.theguardian.com/commentisfree/2024/jan/05/metaverse-sexual-assault-vr-game-online-safety-meta). Many victims report having contemplated suicide (https://www.nytimes.com/2024/03/23/opinion/deepfake-sex-videos.html). This incorrect narrative that sexual digital forgeries do not cause real harm will fuel some perpetrators, giving ‘justification’ in their own minds that creating and sharing digital sexual forgeries is an acceptable thing to do.
2.11. There is an urgent need to change this narrative. In fact, one study found that 74% of those who’ve engaged with deepfake pornography feel no guilt about doing so (https://www.securityhero.io/state-of-deepfakes/#deepfake-porn-survey) implying, at best, ignorance to the harms caused and, at worst, an entrenched misogyny that places women as lower-class citizens deserving of this abuse.
3. What is being done about sexual digital forgeries?
3.1. Lawmakers are starting to recognise the increasing prevalence of sexual digital forgeries and the harms caused, as evidenced by the creation of related legislation in various pockets of the globe. Of note, the US recently introduced the Take It Down Act which “prohibits the nonconsensual online publication of intimate visual depictions of individuals, both authentic and computer-generated, and requires certain online platforms to promptly remove such depictions upon receiving notice of their existence.” (https://www.congress.gov/bill/119th-congress/senate-bill/146).
3.2. Other countries have enacted similar legislation; Australia and France are amongst a handful of nations that have made the sharing of sexual digital forgeries illegal, while South Korea has made it illegal to view or possess deepfakes that are deemed to be sexually exploitative, and China has illegalised the creation and sharing of any deepfake content without the subject’s consent.
3.3. In England and Wales, the Online Safety Act was amended in 2023 to make the sharing of intimate images, including deepfakes, illegal (https://www.legislation.gov.uk/ukpga/2023/50?view=extent) and the Data (Use and Access) Bill has been amended to illegalise the creation and soliciting of “digitally created sexually explicit photographs or films” (https://bills.parliament.uk/publications/57373/documents/5502) - these amendments are currently at royal ascent.
3.4. The legal reform is positive and begins to tackle issues around acceptability, yet there remain many challenges: 1) digital sexual abuse is an online phenomenon, legality varies by jurisdiction and many will be posting content from countries/regions where the creation and sharing of such content is not illegal; 2) definitions of what constitutes ‘sexually explicit’ may vary by jurisdiction; 3) many individuals may be unaware that they’ve been targeted; 4) it can be difficult to remove content online and the digital form of this content means that it is impossible to guarantee content has gone forever leaving victims with an ongoing fear that it will reappear; 5) police resource and experience to deal with reporting of such crimes are likely to create a barrier for victims — it is vital to avoid secondary victimisation as has been evidenced with physical sexual violence against women and girls (Brooks & Burman, 2016).
3.5. Big tech responses have been minimal. In 2024, Google claimed it would decrease the visibility of sexual digital forgeries by lowering this kind of content within search rankings (https://mashable.com/article/google-deepfake-porn-search-results). However, the sites remain discoverable, and a search for ‘deepfake porn’ on Google on the 23rd of June 2025 listed sites hosting this content on the third page of search results.
3.7. Meta makes substantial income from paid advertising on its platforms promoting ‘nudify’ apps, as do Telegram and X (https://www.cbsnews.com/news/meta-instagram-facebook-ads-nudify-deepfake-ai-tools-cbs-news-investigation/). Other social media platforms work on a similar model whereby they generate substantial income from paid advertising—therefore this issue is likely occurring across most, if not all, social media sites. Many of these ads contain a URL that redirects users to Apple's app store or Google’s Play store, where ‘nudify’ apps are available to download. Although company policies and guidelines prohibit the sharing and promotion of non-consensual nudity, the content prevails, implying that a significant increase in efforts is required from the tech giants.
3.8. ‘Nudify’ apps and websites have no or limited age verification, users only need click ‘accept’ on the age warning prompt—this is true both for the age of the user and the age of the target portrayed in the image. Of further note here: although the generated content might be defined as illegal, the technology to create this content is not.
3.9. The free service StopNCII uses hash-matching to detect the non-consensual sharing of intimate images, though focuses primarily on ‘revenge porn’ which typically involves the sharing of real, rather than digitally created/manipulated, content. Services such as this are a positive step although most (if not all) require women and girls to already know this content exists, and to have or acquire a copy of the content in order to submit it to the service (e.g., Stop NCII). As such, these services find exactly or closely matching copies of known NCII but are less helpful for detecting sexual digital forgeries that have been created without the target’s knowledge.
3.10. OFCOM have announced that they will consult on measures involving automated tools for taking down NCII—the outcome is currently unknown.
4. How realistic is AI-generated content?
4.1. There is limited empirical work that directly examines issues relating to sexual digital forgeries, in part due to the ethical challenges surrounding this work and the availability of funding.
4.2. More generally, research shows that people have a very limited ability to accurately distinguish between real and AI-generated facial images (McGuire et al, 2024; Miller et al., 2024; Nightingale & Farid, 2022).
4.3. Further, evidence from training approaches shows that it is difficult to make people better at distinguishing between real and AI-generated facial images (Nightingale & Farid, 2022).
4.4. Whether we can extrapolate these findings to sexual digital forgeries is as yet unknown. But we do know that the harm caused by such content is real regardless of its provenance. Where these findings do have implications is for future interventions and protection strategies—any approaches should appreciate the perceptual realism of the forged content.
5. Current work in the Nightingale Lab (our strategy)
5.1. Novel deepfake technology poses a serious and imminent threat to individuals and society and work is urgently needed to better protect ordinary people.
5.2. We are currently working on several approaches drawing from psychology, computer science, and law to better protect ordinary people from the harms that deepfake technologies present. Some examples are listed below.
Gathering empirical data:
o on emerging deepfake-enabled crimes and what this means for policy and legislation
o to demonstrate the harmful impacts of sexual digital forgeries, including victim perspectives on what is needed in terms of detection and aftercare
o to estimate prevalence of sexual digital forgeries, awareness of legal changes, and understanding of why people create/share such content
o to understand the strengths and weaknesses within law enforcement and prosecution for supporting those who report these crimes
o to gain insight about who the most vulnerable groups/communities are and the impact of intersectionality
Conducting interviews with tech experts to gain horizon-scanning insight into this space to support more proactive protection approaches.
Working with the police, the public, government, and technology experts to co-design and develop FORTRESS—an innovative forensically assured verification system to detect sexual digital forgeries and support those who have been targeted. This system will search for sexual digital forgeries on behalf of ordinary people, and we will work with charities and other organisations to communicate content discovery, support reporting, and remove content—this approach means that individuals do not have to trawl through sites searching for harmful content themselves. FORTRESS aims to put power and control back into the hands of the victim. It is vital to empower women and girls to ensure they are not ultimately forced to remove online presence.
6. Recommendations
6.1. Sexual digital forgeries exist because there are tools to create such content and an audience to consume it. As the epidemic of sexual violence against women and girls continues to grow, we need to find ways to maintain some control over the spread of online misogyny promoted by such content, the following needs to happen:
6.2. Academic research is required to examine the scale and prevalence of AI-enabled sexual abuse, to gain a deeper understanding of the impact this content is having on those targeted and society at large, and to develop evidence-based approaches for victim support.
6.3. Academic research is required to gain insight into the pathways to offending, e.g., how and why are people/teenage boys engaging in this type of crime and the role of the manosphere.
6.4. A much more comprehensive education strategy is essential, including: 1) educational reform from a very young age to highlight the harms associated with this content and the consequences for those creating/sharing it (https://committees.parliament.uk/oralevidence/16148/pdf/) and 2) education for wider society to raise awareness of sexual digital forgeries and their harms.
6.5. More pressure on social media/search engines/video hosting platforms and other websites to prevent sharing of such content and to provide improved reporting mechanisms.
6.6. More general legislation that can be sensibly flexed to keep up with evolving tech. The use of generative AI to create sexual digital forgeries will not be the last form of tech-facilitated VAWG. As the technology continues to evolve, so too will the harms to women and girls—bespoke legal reform with each instance of tech-enabled VAWG will be slow, costly, and leave many vulnerable to being targeted.
6.7. Trust in law enforcement and prosecution must improve. With the recent legal changes in England and Wales, it is vital to ensure that law enforcement is equipped to deal with the reporting of sexual digital forgeries. If not, this will be another instance of women and girls being let down by the system and will fuel further distrust.
6.8. A much wider repertoire of support mechanisms for victims, which include recognition of individual differences in how victims respond to being targeted.
6.9. Greater support for taking down digital sexual forgeries, and a much faster process for removal.
6.10. Stronger regulation of big tech and a clearer distinction between the tech companies and government in regulation planning.
6.11. An ethical and responsible innovation approach to technological development.
7. Further questions for consideration
7.1. Empirical research in this area is limited yet vital to inform policy and provide an improved appreciation of the scale and impact of abuse. A key question from a research perspective is: as legislation changes, how can researchers be protected to legally conduct work in this space?
7.2. The government aim to halve VAWG in a decade. Given the scale of emerging tech-enabled victimisation of women and girls, is it possible that a decrease in reporting might reflect women and girls’ further decreased trust in the law enforcement and prosecution system rather than reduced crime rates?
7.3. Given the phenomenal realism of AI-generated content, how are sites such as Pornhub—which amassed almost 12 billion visits in the month of January 2024 (https://www.statista.com/statistics/1459714/pornhub-monthly-visits-by-device/)—ensuring their content is legal, consensual material?
June 2025