TikTok Sued in Brazil Over AI Moderation Failures Allowing Racist Content

Thumbnail Image

The information displayed in the AIM should not be reported as representing the official views of the OECD or of its member countries.

Brazil's Federal Public Ministry (MPF) filed a lawsuit against TikTok's parent company, ByteDance Brasil, seeking R$ 200 million in damages. The action follows failures of TikTok's AI-based moderation systems to detect and remove racist comments, allowing discriminatory content to persist despite user and official reports.[AI generated]

Why's our monitor labelling this an incident or hazard?

The TikTok platform uses AI-based content moderation systems to detect and remove harmful content, including racist comments. The investigation found that these systems failed to remove racist comments even after being notified, allowing harmful content to remain accessible and cause societal harm. This constitutes a violation of human rights and collective moral harm, fulfilling the criteria for an AI Incident. The event is not merely a potential risk but documents realized harm due to the AI system's malfunction or inadequacy. The legal action and demands for system improvements further confirm the central role of the AI system in the harm caused.[AI generated]
AI principles
FairnessRespect of human rights
Industries
Media, social platforms, and marketing
Affected stakeholders
Consumers
Harm types
Human or fundamental rightsPsychological
Business function:
Monitoring and quality control
AI system task:
Other

Articles about this incident or hazard