Meta Platforms Hosted Thousands of AI 'Nudify' App Ads Promoting Non-Consensual Explicit Images

Thumbnail Image

The information displayed in the AIM should not be reported as representing the official views of the OECD or of its member countries.

Meta's Facebook and Instagram platforms ran thousands of ads for AI-powered 'nudify' apps, primarily placed by Chinese firm GatherOne. These apps use AI to generate non-consensual explicit images, violating Meta's policies and causing harm to individuals. The incident highlights failures in content moderation and enforcement on major social media platforms.[AI generated]

Why's our monitor labelling this an incident or hazard?

The AI systems (nudify apps) are explicitly described as generating non-consensual sexualized deepfake content, including flagged child pornography, which is a clear violation of rights and causes harm to individuals and communities. The ads promoting these apps were served on Meta's platforms, leading to the dissemination of harmful AI-generated content. This direct link between AI system use and realized harm meets the criteria for an AI Incident. The report also highlights failures in enforcement and oversight, reinforcing the incident classification rather than a mere hazard or complementary information.[AI generated]
AI principles
Respect of human rightsPrivacy & data governance

Industries
Media, social platforms, and marketing

Affected stakeholders
General public

Harm types
Human or fundamental rightsPsychological

AI system task:
Content generation


Articles about this incident or hazard