AI-Generated Fake Nudes of Schoolgirls Spark Police Investigations in Brazil

Thumbnail Image

The information displayed in the AIM should not be reported as representing the official views of the OECD or of its member countries.

Students in schools across Recife, Rio de Janeiro, and Belo Horizonte used AI tools to create and distribute fake nude images of female classmates, many of them minors. The incidents led to police investigations, emotional distress for victims, and raised concerns about privacy violations and the misuse of AI for image manipulation.[AI generated]

Why's our monitor labelling this an incident or hazard?

The use of a deep nude app, which employs AI to generate fake nude images by altering photos, directly led to harm to the victims' mental health and a violation of their rights. The AI system's use in producing and disseminating these manipulated images constitutes an AI Incident as it caused realized harm to individuals (psychological trauma and violation of privacy).[AI generated]
AI principles
Privacy & data governanceRespect of human rightsSafetyHuman wellbeingAccountabilityRobustness & digital security

Industries
Education and trainingDigital securityMedia, social platforms, and marketing

Affected stakeholders
Children

Harm types
PsychologicalHuman or fundamental rightsReputational

Severity
AI incident

AI system task:
Content generation


Articles about this incident or hazard

Thumbnail Image

'Nunca vão saber o trauma que causaram', diz aluna vítima de nudes falsos em colégio no Rio

2023-11-06
Terra
Why's our monitor labelling this an incident or hazard?
The use of a deep nude app, which employs AI to generate fake nude images by altering photos, directly led to harm to the victims' mental health and a violation of their rights. The AI system's use in producing and disseminating these manipulated images constitutes an AI Incident as it caused realized harm to individuals (psychological trauma and violation of privacy).
Thumbnail Image

Alunas de colégio em Recife são vítimas de falsos nudes feitos com IA

2023-11-07
Olhar Digital - O futuro passa primeiro aqui
Why's our monitor labelling this an incident or hazard?
The article explicitly mentions the use of AI to create false nude images of minors, which is a clear violation of human rights and causes harm to the victims. The involvement of AI in generating these images and the resulting harm to the individuals and communities qualifies this event as an AI Incident under the OECD framework. The harm is realized, not just potential, and the AI system's use is central to the incident.
Thumbnail Image

Alunas de colégio particular do Recife são vítimas de falsos nudes; polícia apura o caso

2023-11-07
JC
Why's our monitor labelling this an incident or hazard?
The event explicitly mentions the use of AI applications to create manipulated nude images of minors, which constitutes a violation of rights and harm to individuals. The AI system's use directly led to the harm through the creation and dissemination of false explicit content. Therefore, this qualifies as an AI Incident under the framework, as the AI system's use has directly led to harm to persons and violation of rights.
Thumbnail Image

Alunas de colégio particular do Recife são vítimas de falsos nudes; polícia apura o caso

2023-11-07
Correio
Why's our monitor labelling this an incident or hazard?
The article explicitly mentions the use of artificial intelligence to create false nude images of students, which constitutes a direct harm to the victims through violation of their privacy and potentially their rights. The AI system's use in generating manipulated images that were shared without consent fits the definition of an AI Incident, as it has directly led to harm to individuals (violation of rights and harm to communities).
Thumbnail Image

Jovem suspeito de criar falsos nudes no Colégio Santo Agostinho, em BH, era amigo das vítimas

2023-11-09
itatiaia.com.br
Why's our monitor labelling this an incident or hazard?
The article explicitly states that the suspect used artificial intelligence to create false nude images of classmates and shared them publicly, leading to emotional distress and reputational harm to the victims. This constitutes a violation of rights and harm to individuals, fitting the definition of an AI Incident. The AI system's use was central to the harm caused, as it enabled the creation of realistic fake images that were used maliciously. Therefore, this event qualifies as an AI Incident rather than a hazard or complementary information.
Thumbnail Image

O que se sabe sobre o vazamento de nudes de alunas criados com IA em colégio do Rio

2023-11-06
CartaCapital
Why's our monitor labelling this an incident or hazard?
The use of AI to generate fake nude images of minors constitutes a violation of their rights and privacy, which is a breach of applicable laws protecting children and adolescents. The dissemination of these images causes harm to the individuals and communities involved. Since the AI system's use directly led to this harm, this qualifies as an AI Incident under the framework, specifically under violations of human rights and harm to individuals.
Thumbnail Image

Já ouviu falar em 'Deep Nude'? Mais alunas são vítimas de técnica de manipulação de fotos

2023-11-08
Metro World News Brasil
Why's our monitor labelling this an incident or hazard?
The article explicitly mentions the use of an AI technique called Deep Nude to generate manipulated images of minors without clothing, which were then distributed, constituting a clear violation of rights and harm to the victims. The AI system's use is central to the harm caused, fulfilling the criteria for an AI Incident involving violations of human rights and harm to individuals. Therefore, this event qualifies as an AI Incident.