AI Nude-Generating Apps Bypass Parental Controls, Exposing Minors

Thumbnail Image

The information displayed in the AIM should not be reported as representing the official views of the OECD or of its member countries.

A Tech Transparency Project investigation found that minors could access AI-powered apps generating fake nude images, bypassing parental controls on Apple and Google platforms. Researchers identified 135 such apps, some misclassified for young children, exposing minors to inappropriate content and privacy violations. Apple and Google removed some apps after being alerted.[AI generated]

Why's our monitor labelling this an incident or hazard?

The AI systems in these apps are used to generate or manipulate nude images, which can cause harm to children exposed to such content, violating protections for minors and potentially causing psychological or social harm. The event involves the use of AI systems leading to direct harm (exposure of minors to inappropriate content), fulfilling the criteria for an AI Incident. The companies' removal of apps and policy changes are responses to this incident but do not change the classification of the event described.[AI generated]
AI principles
SafetyPrivacy & data governance

Industries
Media, social platforms, and marketing

Affected stakeholders
Children

Harm types
PsychologicalHuman or fundamental rights

AI system task:
Content generation


Articles about this incident or hazard