AI Misuse: Minors Create Deepfake Images of Peers

Thumbnail Image

The information displayed in the AIM should not be reported as representing the official views of the OECD or of its member countries.

A survey by Thorn reveals that 10% of minors in the U.S. report peers using AI to create deepfake nude images of other children. This misuse of AI technology has led to several student arrests and highlights significant concerns about online safety and child exploitation.[AI generated]

Why's our monitor labelling this an incident or hazard?

The event describes actual misuse of AI systems (generative deepfake tools) to produce intimate images of others without consent, which has led to legal action and constitutes a violation of personal and sexual rights. The AI system’s use directly resulted in harmful, criminal behavior, fitting the definition of an AI Incident.[AI generated]
AI principles
SafetyPrivacy & data governanceRespect of human rightsAccountabilityHuman wellbeingRobustness & digital security

Industries
Education and trainingMedia, social platforms, and marketingDigital securityGovernment, security, and defenceHealthcare, drugs, and biotechnology

Affected stakeholders
Children

Harm types
PsychologicalReputationalHuman or fundamental rights

AI system task:
Content generation

In other databases


Articles about this incident or hazard