Written evidence submitted by the Antisemitism Policy Trust (SMH0005)

The Antisemitism Policy Trust is a charity that works to educate and empower parliamentarians and policy makers to address antisemitism. For more than ten years, the Trust has provided the secretariat to the All-Party Parliamentary Group (APPG) Against Antisemitism. The Trust has advised the government, policy makers and Ofcom about online harms, including in relation to the Online Safety Bill, other legislation and related matters, and continues to do so. We have also carried out extensive research and published briefings on online antisemitism, conspiracy theories and Artificial Intelligence (AI).[1][2]

We welcome the call for evidence from the Science, Innovation and Technology Committee into misinformation and social media. Our submission focuses on the use of AI and the ease with which generative AI can be used to produce content that includes harmful and racist disinformation.

The Threat of Disinformation

The spread of disinformation online can be directly linked to harmful real-life consequences. This includes the erosion of trust in public institutions, an undermining of democratic processes – for example, free and fair elections, and the development of risks to public health. Disinformation can also mobilise people in a way that increases the risk of violence, especially against marginalised communities and minority groups.

The Southport riots, prompted by the spread of online disinformation regarding the identity and motives of the murderer, are just one example of the ability of misinformation and disinformation to inspire violence. There are many other recorded incidents in which violence was triggered by online disinformation, including in the U.S., India, and Myanmar. In some of those cases, people have been killed and injured as a result.[3]

Violent attacks against Jews have been carried out by individuals who in certain cases have been, at least partially, radicalised online and exposed to disinformation. Most of the perpetrators of those attacks – if not all them – believed in antisemitic conspiracy theories which sit at the heart of a significant proportion of the racism against Jews. These anti-Jewish conspiracies allow people to blame Jews for their grievances, and play into preexisting attitudes and fears.

The gunman who killed 11 congregants at a Pittsburgh Synagogue, for example, believed in the Great Replacement theory. According to this conspiracy, Jews have a plan to bring non-white immigrants to Europe and North America in order to carry out a ‘white genocide’ in which white people are replaced by other minority ethnic groups.[4] This belief forms part of a world view based on lies and conspiracies, amplified by online propaganda.

At times, a wave of disinformation is triggered in response to events. The war between Israel and Hamas following the 7 October terrorist attacks, had fed a considerable rise in antisemitic incidents in the UK[5] and in antisemitism online. This has been intensified by the large-scale spread of misinformation, disinformation and conspiracy theories.

It is worth the committee considering how these terms are often used interchangeably - that ultimately for those sharing or engaging in these online behaviours, it doesn’t matter. When does disinformation become hate content, become extremist content, become terrorist content, become misinformation. What it ultimately speaks to is the need for close working collaboration between these areas - from civil society, to police, counter-terrorism experts and social scientists - these issues represent shades of grey and as such they require an understanding of that.

Search Engines and Disinformation

The systems that sit behind search engines have also been complicit in the spread harmful disinformation. Trust Chief Executive Danny Stone, in evidence to the Online Safety Bill committee in 2022, detailed for members that the Microsoft Bing bar, was prompting users/autocompleting the sentence ‘Jews are’ with the term ‘bastards’. Searching for ‘Gays are,’ on Bing returned a  link in the highest ranking order to a story suggesting that “Gays are using windmills to waft homosexual mists into your home.”[6] The same indexing on Google led users to a factual site about homophobia maintained by the Southern Poverty Law Centre. We considered the failure to proactively review potentially harmful prompts for disinformation a failure of due diligence by the corporation.

A study that we published in 2019 together with the Community Security Trust (CST) and researcher Seth Stephens-Davidowitz, studied Google searches emanating from the UK. It revealed that people express hatred of Jews and violent tendencies against Jews in their searches. It also found that about 170,000 Google searches with antisemitic content are made in the UK each year.[7] Searches also revealed a rise in the popularity of antisemitic conspiracy theories. It is likely that those numbers have increased since the study was published.

Perhaps of most relevance to the committee’s call for evidence, we have found that some of the searches produced results that – instead of directing people to factual information and away from content that spreads anti-Jewish hatred and disinformation – reinforced those beliefs by producing top results that support such information.[8]

Our study concluded that ‘When Google altered its algorithm to remove “Are Jews evil?” from its auto-complete function, the number of people making this search fell by ten per cent. This alone should show why technology companies need to play a much more proactive role in finding ways to stop the spread of antisemitism in our society.’[9]

Antisemitism and the Risks of Generative-AI

Artificial Intelligence has added to this barrage of disinformation. AI-generated images and deep fake videos used to manipulate public opinion have surfaced since the start of the war.[10] Some of these have been sophisticated enough to fool social media users, who have shared those images, thinking they represent true depictions of events. Others have shared these knowing that they are fake, but wanting to intensify a particular narrative, including anti-Jewish ideas and streams of thought.  In many instances, these images contained emotive content that has been designed to manipulate viewers and shape their views. The influx of AI-generated disinformation has been so great, that fact checkers and analysts have been struggling to moderate the content.[11]

There are multiple different types of generative AI content being produced, and the situation is complex. Some users are creating and sharing this content for propaganda reasons (i.e. the content is not designed to dupe) but owing to the realistic nature of it, there is a risk that it may still dupe the users viewing it. Then there are those users who are utilising these tools in order to produce content that is designed to deceive. Within all of this is the fact that a lot of this is being facilitated by existing extremist networks and spaces online – the failure sits with both AI companies, and the social media companies that aren’t doing enough to curb the spread of this type of content.

The Online Safety Act needed to be equipped to be dynamic and adaptable. At present it is unwieldy and slow, which doesn’t reflect the nature of the problem it is trying to address.

A recent study by that we published, together with  Project Decoding Antisemitism and INACH (the international network for combating online hate) found that whilst existing ‘classifiers’ or systems for labelling AI can detect deepfakes to a reasonable extent, they struggle to find and classify antisemitic deepfakes.[12]

Our findings show that current algorithmic solutions struggle to account for complex, nuanced forms of imagery, which are particularly prevalent in the dissemination of hate ideologies. As online actors try to avoid automatic recognition, they often resort to implicit rather than explicit, obvious patterns, making detection even more challenging.

AI-generated antisemitic content, which can be nuanced and is constantly evolving, propagates harmful stereotypes and inspires hatred of  Jews. This in turn contributes to the rise in antisemitism, exposing Jewish people to threats and harassment online and in real world scenarios. It also poses a serious threat to social cohesion and public safety, including increased risk of terror attacks. Given the spread of such images, and the continual advancement of the technologies behind them,  we contend that more needs to be done to fix the systems behind, and approaches to, generative AI.

In relation to the Committee's question about the role of generative AI and LLMs, in our aforementioned study, we found with relative ease, AI-generated images including antisemitic themes, for example Nazism and support for terrorism. This demonstrates the simplicity of circumventing the existing safety measures taken by those AI companies permitting users to generate content. Some companies incorporate safety features in their generative AI tools. However, many of these features can be manipulated to produce racist content by using clever, even simple prompts. For example, large language models (LLM) have been gamed to produce the desired results. In our AI briefing, we explained that online actors boasted a work around for producing images of Hitler. Whilst doing so was blocked by many companies, asking generative AI to produce an image of a ‘WW2 German Chancellor’ generated the desired image.[13]

Once radicalising content has been created, it can be uploaded onto social media platforms and receive widespread exposure, because the business model of some platforms still relies on the  engagement produced by an increased exposure of divisive and extreme content. Some of this material includes disinformation about Jews or about the war between Israel and Hamas, and much of it fosters hatred of Jews. Not all of the AI-generated content that we have found was hyper-realistic, some was clearly fake, including animated images, but much of it has been used to foster age-old tropes about Jews.

To give an example of the nature of the content we are describing, one image found by our researchers contains a hidden antisemitic caricature of ‘the happy merchant’ – the most widespread antisemitic meme. It is a grotesque stereotype representing a hook-nosed Jewish man gleefully rubbing his hands. The merchant is visible if the image below is made smaller and seen from a distance. It consists of two rats in a bin. Jews have been historically been described as filthy vermin, and their depiction as rats has also been used in Nazi propaganda:

AI Generated Image                                          Original Image

CST also found that users on the platform 4chan, a small, high harm online forum, asked generative AI told to produce an image of a ‘Jew about to be killed, afraid, screaming.’ The image that resulted from the prompt included antisemitic stereotypes to satisfy the ‘Jewish’ appearance of the character in the image.’[14]

AI powered voice systems can also produce harmful, antisemitic disinformation. The Trusts’s Chief Executive explained to the Online Safety Bill Public Bill Committee that when asking Amazon’s Alexa ‘Is George Soros Evil?’, the response he received was ‘Yes, he is.’ More disinformation was provided by Alexa to the question ‘Are the White Helmets fake?’ The reply was ‘Yes, they are set up by an ex-intelligence officer.’[15] In these instances, the information gathering processes were highlighted as insufficient and not quality assured. With exposure to millions of homes, the technology behind such products must be failsafe. Alexa has in the past been proven to give antisemitic answers in response to prompts.[16]

The Case for Regulating AI

As AI advances, images and ‘deepfake’ videos become increasingly realistic, making content more ‘believable’ and the dissemination of disinformation easier. Effective AI governance therefore requires more robust oversight, safety detection features and ethical guidelines for the creation of AI content. This is something that generative AI companies, social media services, and search engines should all be part of.

Social media platforms have, for far too long, neglected to implement ‘safety by design’ features with the ability to reduce the spread and visibility of harmful information. Their algorithms have been designed to amplify content that promotes engagement – and therefore, revenue for the services. In many cases, this content is extreme, including racist and misogynistic materials, and other themes with a propensity to arouse strong emotions, including fear and anger.

Extremists know how to use platforms’ algorithm to spread their views, including for example luring other users away from mainstream, reliable, sources of information to extremist content providers that often publish fake news.[17] 

A new study by the Antisemitism Policy Trust about the meteoric rise of online antisemitism after 7 October 2023, highlights the use, for example, of ‘thread hijacking’ by alt-influencers.[18] This is a practice in which extreme and antisemitic accounts use the comments section of celebrity posts that receive high engagement, to raise their profile, spread their own messages and direct other users to extremist content that often contains disinformation and antisemitism.

Our research found a 15-fold increase in antisemitism on X. Between October 2023 and July 2024 we identified over 129,200 posts containing explicit antisemitism, out of a sample of 4.95 million posts. Some of these have been removed, but thousands of others remain online and more appear each day. The number of ‘likes’ of such posts has also jumped 1,3000-fold  from 18,741 in October 2023 to over 24 million by June 2024. The number of retweets jumped 1,900-fold in that period. Our algorithms managed to easily identify antisemitic posts on mainstream and other platforms, raising the question why some large social media services do not use effective existing commercial safety systems.

In direct regard to the Committee’s question about platforms business models, one of the principles on which social media platforms rely, is that of ‘Priming.’ ‘Priming’ means that information consumed by a person affects their behaviour, especially if the exposure is repetitive. Through priming, people will start expressing more extreme views for example. They could also be incited to violence or rioting because of exposure to large amounts of ideologically-motivated disinformation, or made to spend more money on brands and products, all of which benefits the platform.

Extremist mis- and disinformation not only has the ability to radicalise people into extremist violence by reinforcing existing beliefs. It can influence people who do not actively seek out this information, but who have been exposed to it due to lack of safety mechanisms employed by social media companies. In 2021 we conducted a study in collaboration with the Community Security Trust (CST) and the Woolf Institute about antisemitism on Instagram. We found that antisemitic content was suggested to users even if they had not previously looked for it.[19]

Through its various clauses, the Online Safety Act (OSA) is argued to be reducing illegal harms and the spread of disinformation. The OSA includes several measures to tackle disinformation including a False Communications offence and a requirement for platforms to address disinformation seeded by foreign states.

However, in relation to your question about the effectiveness of the Act, it has been suggested that the False Communications offence may be too vague and difficult to enforce at scale and that the bill focuses on specific offences rather than the broader harms caused by mis- and disinformation.[20] Furthermore, we were disappointed that the committee on mis- and disinformation, to be established by Ofcom, is yet to be convened and will have limited (if any) powers. We therefore urge the Committee to consider recommending to Government a more comprehensive plan to tackle disinformation and its wider harms.

Certainly, enhanced transparency is required. Social media platforms and companies that generate AI, should ensure that AI-generated content is labelled as such. This way, content can remain online as long as it is legal, but users will be aware that what they are seeing is artificial and not real.

Social media services should also employ better tools that are able to identify disinformation that can cause harms, including racism and incitement to violence. When this content is legal, platforms’ terms and services will be the determinant factor, but their algorithms should not automatically promote this content without built in analysis and safety systems, offering free exposure and amplification. Preferably, where hateful deepfakes are involved, social media services should label such content as potentially offensive or warn that it may contain false information and direct users to reliable sources of information.

A further important measure is the ability to trace AI-generated content to its source – the service used to create it. AI companies whose tools were used to make illegal content might be made liable for failing to prevent it from being produced. This could serve as an incentive for the adoption of improved safety features.

It is vital to strike a balance between users’ safety and public order by reducing exposure to harmful disinformation, and maintaining freedom of expression, which is one of the building blocks of our democracy. The Antisemitism Policy Trust is calling not for impediments to freedom of expression but enhanced transparency, user-empowerment and stronger barriers to the creation and dissemination of illegal content.

Alongside its disadvantages and risks, Artificial Intelligence has many benefits, including the ability to process vast amounts of information quicker than any human could. It has many potential uses across a large number of sectors. However, AI is only as intelligent as the people that design and inform it. Understanding the potential gaming and manipulation of AI is an imperative.

This is a global issue. We urge the Government, and parliamentarians to work with international partners in order to establish ethical standards for the use of AI. Forming guidelines for the safe use of generative AI and requiring social media platforms and search engines to have better control over the spread of mis- and disinformation and to offer greater transparency in relation to AI-generated content, are key to enhancing online and offline safety.

17 December 2024