TikTok's AI Algorithm Criticized for Promoting Harmful Content and Censorship

Thumbnail Image

The information displayed in the AIM should not be reported as representing the official views of the OECD or of its member countries.

TikTok's AI-driven recommendation and moderation systems have been criticized for promoting content that glamorizes eating disorders and for allegedly suppressing posts from minority creators. These issues have led to harm to users' mental health and violations of rights, highlighting the negative impact of TikTok's AI algorithms on its community.[AI generated]

Why's our monitor labelling this an incident or hazard?

TikTok's AI-powered recommendation algorithm is explicitly involved in the event, as it suggests content that can trigger or worsen eating disorders. The harm is to users' mental health, a form of injury or harm to health, which has materialized as described by users and campaigners. The AI system's use (content recommendation and moderation) has indirectly led to this harm by promoting or failing to adequately remove harmful content. This fits the definition of an AI Incident because the AI system's development and use have directly or indirectly caused harm to people's health. The article does not merely warn of potential harm but reports ongoing harm experienced by users, thus it is not an AI Hazard or Complementary Information. It is not unrelated because the AI system is central to the issue.[AI generated]
AI principles
FairnessHuman wellbeingRespect of human rightsSafetyTransparency & explainabilityDemocracy & human autonomyAccountability

Industries
Media, social platforms, and marketing

Affected stakeholders
Consumers

Harm types
PsychologicalHuman or fundamental rights

Business function:
Monitoring and quality control

AI system task:
Organisation/recommendersRecognition/object detection


Articles about this incident or hazard