AI-Generated Deepfake of Missing Child Causes Distress and Misinformation in Florence

Thumbnail Image

The information displayed in the AIM should not be reported as representing the official views of the OECD or of its member countries.

AI-generated deepfake videos featuring missing child Kata circulated on TikTok, using synthetic voice and animation to mimic her appeal for help. The videos misled viewers, caused emotional distress, and spread misinformation, prompting calls from experts for regulation of such technology due to its potential for harm.[AI generated]

Why's our monitor labelling this an incident or hazard?

The event explicitly describes an AI system generating a fake video and voice of a missing child, which is being disseminated and believed by some users, causing emotional harm and misinformation. This fits the definition of an AI Incident because the AI system's use has directly led to harm to communities and individuals (emotional distress, misinformation). The harm is realized, not just potential, and the AI system's role is pivotal in creating the false content. Therefore, this is classified as an AI Incident.[AI generated]
AI principles
Transparency & explainabilityPrivacy & data governanceRespect of human rightsSafetyAccountabilityRobustness & digital securityHuman wellbeing
Industries
Media, social platforms, and marketingDigital securityGovernment, security, and defence
Affected stakeholders
ChildrenGeneral public
Harm types
PsychologicalHuman or fundamental rightsReputational
Severity
AI incident
AI system task:
Content generation

Articles about this incident or hazard