TikTok's AI Algorithms Enable Spread of Antisemitic and Hateful Content Among Youth

Thumbnail Image

The information displayed in the AIM should not be reported as representing the official views of the OECD or of its member countries.

The Bildungsstätte Anne Frank criticizes TikTok for enabling the rapid spread of antisemitic, racist, and conspiratorial content among young users through its AI-driven recommendation and moderation systems. AI-generated images and hate speech are widely disseminated, contributing to the radicalization and harm of vulnerable communities.[AI generated]

Why's our monitor labelling this an incident or hazard?

TikTok's AI-powered recommendation system is directly involved in distributing harmful content that radicalizes young users and spreads antisemitic misinformation. The use of AI-generated images further implicates AI in the creation and spread of harmful content. This has led to realized harm to communities and potentially violates rights, meeting the criteria for an AI Incident. The article reports on ongoing harm rather than a potential or future risk, so it is not an AI Hazard or Complementary Information.[AI generated]
AI principles
FairnessRespect of human rightsSafetyAccountabilityTransparency & explainabilityDemocracy & human autonomy

Industries
Media, social platforms, and marketing

Affected stakeholders
ChildrenOther

Harm types
Human or fundamental rightsPsychologicalPublic interest

Severity
AI incident

Business function:
Marketing and advertisementMonitoring and quality control

AI system task:
Organisation/recommendersEvent/anomaly detection


Articles about this incident or hazard

Thumbnail Image

Verbreitung von Antisemitismus und Rassismus: Frankfurter Bildungsstätte kritisiert TikTok

2024-02-05
T-online.de
Why's our monitor labelling this an incident or hazard?
While TikTok likely uses AI recommendation systems that influence content exposure, the article focuses on the platform's role in spreading harmful content without explicitly linking this to AI system development, use, or malfunction. The harm (spread of antisemitism and racism) is occurring, but the AI system's role is indirect and not clearly established as pivotal in this report. Therefore, this is best classified as Complementary Information, as it provides context and criticism related to AI-driven platforms but does not document a specific AI Incident or Hazard.
Thumbnail Image

Kritik von Frankfurter Bildungsstätte: "TikTok radikalisiert Jugendliche"

2024-02-05
Bild
Why's our monitor labelling this an incident or hazard?
TikTok's AI-powered recommendation system is directly involved in distributing harmful content that radicalizes young users and spreads antisemitic misinformation. The use of AI-generated images further implicates AI in the creation and spread of harmful content. This has led to realized harm to communities and potentially violates rights, meeting the criteria for an AI Incident. The article reports on ongoing harm rather than a potential or future risk, so it is not an AI Hazard or Complementary Information.
Thumbnail Image

Antisemitismus: Bildungsstätte Anne Frank kritisiert TikTok für "Speed-Radikalisierung"

2024-02-06
Spiegel Online
Why's our monitor labelling this an incident or hazard?
TikTok's content recommendation system is an AI system that influences what users see. The criticism highlights that this AI-driven system is indirectly causing harm by exposing vulnerable young people to hateful and extremist content, leading to rapid radicalization. This constitutes harm to communities, fitting the definition of an AI Incident due to the AI system's indirect role in causing social harm through its content curation and recommendation functions.
Thumbnail Image

Soziale Medien: Verbreitung von Israelhass: Bildungsstätte kritisiert Tiktok

2024-02-05
ZEIT ONLINE
Why's our monitor labelling this an incident or hazard?
The article explicitly mentions AI-generated images being used to spread antisemitic motives and misinformation on TikTok, which is a social media platform employing AI systems for content generation and recommendation. The spread of such harmful content has directly led to harm to communities through radicalization and hate speech. The AI system's role in generating and disseminating this content is pivotal to the harm described. Hence, this qualifies as an AI Incident under the framework, as it involves realized harm caused by AI system use.
Thumbnail Image

Verbreitung von Israelhass: Bildungsstätte kritisiert Tiktok - WELT

2024-02-05
DIE WELT
Why's our monitor labelling this an incident or hazard?
The article explicitly mentions AI-generated images being shared on TikTok to promote antisemitic motives, which directly contributes to harm to communities through hate speech and misinformation. The AI system's use in generating harmful content that is widely disseminated and radicalizes vulnerable groups meets the criteria for an AI Incident. The harm is realized, not just potential, as the content is actively influencing and harming communities. Hence, the event is classified as an AI Incident.
Thumbnail Image

Bildungsstätte kritisiert Tiktok wegen Israelhass - "Digitale Streetworker" gefordert - WELT

2024-02-06
DIE WELT
Why's our monitor labelling this an incident or hazard?
TikTok is an AI-driven social media platform that uses AI algorithms to recommend and amplify content. The article highlights that antisemitic hate speech and AI-generated images are being widely shared on TikTok, leading to radicalization and harm to vulnerable groups, especially youth. The AI system's role in content generation and dissemination directly contributes to violations of rights and harm to communities. Hence, this event meets the criteria for an AI Incident as the AI system's use has directly led to significant harm.
Thumbnail Image

"Verstörender Content für vulnerable Gruppe": Tiktok radikalisiert Nutzer Studie zufolge "über Nacht"

2024-02-05
Der Tagesspiegel
Why's our monitor labelling this an incident or hazard?
TikTok's content recommendation system is an AI system that curates and promotes videos to users based on complex algorithms. The study shows that this AI-driven system is directly linked to the spread of antisemitic and extremist content, leading to rapid radicalization among young users. This constitutes harm to communities and a violation of rights, as vulnerable groups are targeted and exposed to hate speech and misinformation. Since the harm is occurring and the AI system's role is pivotal in content dissemination and radicalization, this event qualifies as an AI Incident.
Thumbnail Image

Radikalisiert TikTok die Jugend? Experten warnen!

2024-02-05
TAG24
Why's our monitor labelling this an incident or hazard?
TikTok's AI-powered recommendation system is implicated in the spread of harmful content that radicalizes young users and disseminates antisemitic and racist material. The use of AI-generated images to fuel hate further confirms AI involvement. The harm to communities through radicalization and misinformation is occurring, making this an AI Incident as the AI system's use directly leads to societal harm.
Thumbnail Image

Hass und Falschinformationen: Bildungsstätte Anne Frank kritisiert TikTok

2024-02-06
Frankfurter Rundschau
Why's our monitor labelling this an incident or hazard?
TikTok's platform uses AI systems (algorithms) to moderate and recommend content. The article highlights that these AI systems are not sufficiently effective in preventing the spread of harmful content such as hate speech and misinformation, which has led to real-world harms including radicalization and the spread of antisemitic and racist content. This constitutes harm to communities and violations of rights. Since the harm is occurring and the AI system's malfunction or inadequacy is a contributing factor, this qualifies as an AI Incident.
Thumbnail Image

Bildungsstätte Anne Frank kritisiert TikTok

2024-02-06
Vorarlberg Online
Why's our monitor labelling this an incident or hazard?
TikTok is a social media platform that uses AI systems for content recommendation and moderation. The article highlights that AI-generated images are being spread on TikTok to promote antisemitic views, which contributes to the radicalization of young people and harms communities. This constitutes an AI Incident because the AI system's outputs (content generation and recommendation) have directly led to harm to communities through the spread of hateful and manipulative content. The event also includes calls for stronger regulation and digital education, but the primary focus is on the realized harm caused by AI-enabled content on the platform.
Thumbnail Image

Bildungsstätte Anne Frank: Tiktok verbreitet Israelhass, Antisemitismus und Verschwörungserzählungen

2024-02-05
HAZ – Hannoversche Allgemeine
Why's our monitor labelling this an incident or hazard?
TikTok uses AI algorithms to recommend and amplify content, including AI-generated images. The spread of antisemitic and racist content and disinformation facilitated by these AI-driven recommendation systems has directly led to harm to communities by fostering radicalization and hate. Therefore, this qualifies as an AI Incident due to the realized harm caused by the AI system's role in disseminating harmful content and influencing vulnerable populations.
Thumbnail Image

Antisemitische Radikalisierung über Nacht bei TikTok

2024-02-05
Westfälische Nachrichten
Why's our monitor labelling this an incident or hazard?
TikTok's content recommendation system is an AI system that influences what users see. The study indicates that this AI-driven platform is enabling the spread of antisemitic and hateful content, leading to real-world harm in the form of radicalization and hate speech among youth. This constitutes a violation of human rights and harm to communities, fitting the definition of an AI Incident due to the AI system's indirect role in causing harm through its content curation and recommendation mechanisms.
Thumbnail Image

Verbreitung von Israelhass: Bildungsstätte kritisiert Tiktok

2024-02-05
op-online.de
Why's our monitor labelling this an incident or hazard?
The article explicitly mentions AI-generated images being used to spread antisemitic content on TikTok, which is a platform employing AI for content recommendation and moderation. The sharing of such AI-generated hateful content has directly led to harm to communities by promoting antisemitism and racism. Although the article focuses on the broader phenomenon rather than a single discrete event, the ongoing dissemination of harmful AI-generated content constitutes an AI Incident under the framework, as it involves realized harm to communities through AI system use.