AI Deepfake Apps Used to Create and Circulate Fake Nude Images of Minors, Prompting Legislative Action in US States

Thumbnail Image

The information displayed in the AIM should not be reported as representing the official views of the OECD or of its member countries.

AI-powered apps were used by a male student at Issaquah High School to generate and share fake nude images of female classmates, causing psychological harm and rights violations. The incident spurred Washington and other US states to pass laws banning the creation and distribution of AI-generated sexually explicit images of minors.[AI generated]

Why's our monitor labelling this an incident or hazard?

The article explicitly mentions the use of an AI app designed to generate fake nude images of real girls, which were then circulated among students, causing harm to the individuals depicted and their community. This constitutes a violation of rights and harm to communities, fitting the definition of an AI Incident. The legislative action is a complementary response but does not change the classification of the core event as an AI Incident.[AI generated]
AI principles
AccountabilityPrivacy & data governanceRespect of human rightsSafetyTransparency & explainability

Industries
Consumer services

Affected stakeholders
Children

Harm types
PsychologicalHuman or fundamental rights

Severity
AI incident

AI system task:
Content generation

In other databases

Articles about this incident or hazard

Thumbnail Image

Spurred by Teen Girls, States Move to Ban Deepfake Nudes

2024-04-22
The New York Times
Why's our monitor labelling this an incident or hazard?
The article explicitly mentions the use of an AI app designed to generate fake nude images of real girls, which were then circulated among students, causing harm to the individuals depicted and their community. This constitutes a violation of rights and harm to communities, fitting the definition of an AI Incident. The legislative action is a complementary response but does not change the classification of the core event as an AI Incident.
Thumbnail Image

Washington state cracks down on AI porn

2024-04-23
Daily Mail Online
Why's our monitor labelling this an incident or hazard?
The article explicitly mentions an AI app used to create fake sexually explicit images of real minors, which were shared and caused trauma and harm to the victims. This fits the definition of an AI Incident because the AI system's use directly led to harm (violation of rights and psychological harm). The legislative responses are complementary information but do not change the classification of the core event. The presence of AI is clear, the harm is realized, and the event involves the use of AI systems to generate harmful content, meeting the criteria for an AI Incident.
Thumbnail Image

States Rush to Combat AI-Generated Deepfake Nudes Targeting Minors

2024-04-23
Breitbart
Why's our monitor labelling this an incident or hazard?
The article clearly identifies AI systems (AI-powered deepfake generation apps) being used to create and distribute harmful content depicting minors, constituting child sexual abuse material. This has led to direct harm to individuals (minors), including violations of their rights and legal consequences for offenders. The AI system's use is central to the harm, fulfilling the criteria for an AI Incident. The legislative responses and arrests are reactions to this incident, not the primary event itself.
Thumbnail Image

Spurred by teen girls, US states move to ban deepfake nudes

2024-04-25
The Star
Why's our monitor labelling this an incident or hazard?
The event involves the use of AI systems to generate sexually explicit deepfake images of minors without consent, which has directly caused harm to the victims, including mental and reputational harm, and constitutes a violation of rights. The article reports on actual incidents of harm caused by these AI-generated images and the legislative responses to address these harms. Therefore, this qualifies as an AI Incident because the AI system's use has directly led to significant harm and rights violations.
Thumbnail Image

Spurred by teen girls, WA and other states move to ban deepfake nudes

2024-04-22
The Seattle Times
Why's our monitor labelling this an incident or hazard?
The event involves the use of AI systems (AI-powered apps that generate fake nude images) to create sexually explicit deepfake images of real minors without consent. This use has directly caused harm to the victims, including mental and reputational harm, and constitutes a violation of rights. The article reports on actual incidents where these harms occurred, as well as legislative responses to address these harms. Therefore, this qualifies as an AI Incident under the framework because the AI system's use directly led to harm to individuals and violations of rights.
Thumbnail Image

John and Shari: Sen. Mullet speaks on deepfakes, run for governor

2024-04-24
My Northwest
Why's our monitor labelling this an incident or hazard?
The event involves the use of an AI system (an AI app that 'strips' clothing from images to create fake nude photos) which directly led to harm to individuals (victims of non-consensual AI-generated explicit images). This harm falls under violations of rights and harm to communities. The article details the incident, the harm caused, and the legislative response, indicating that the AI system's use has directly led to harm. Therefore, this qualifies as an AI Incident.