Lawsuit Expands Over AI-Generated Child Sexual Abuse Material via Grok and Stable Diffusion

Thumbnail Image

The information displayed in the AIM should not be reported as representing the official views of the OECD or of its member countries.

A class-action lawsuit against xAI and Stability AI alleges their generative AI tools, Grok and Stable Diffusion, were used to create thousands of deepfake child sexual abuse images from real photos of minors. Plaintiffs claim inadequate safeguards enabled this misuse, causing severe psychological harm and obstructing investigations. The incident centers on the U.S.[AI generated]

Why's our monitor labelling this an incident or hazard?

The event clearly involves AI systems (Grok and Stability AI models) used to generate illegal and harmful content (CSAM). The AI system's malfunction or inadequate safeguards directly led to the creation and dissemination of harmful material, causing injury and psychological harm to minors (harm to health and communities). Additionally, the alleged obstruction of law enforcement investigations constitutes a violation of legal obligations. These factors meet the criteria for an AI Incident, as the AI system's development, use, and malfunction have directly led to significant harm and legal violations.[AI generated]
AI principles
SafetyRespect of human rights

Industries
Media, social platforms, and marketing

Affected stakeholders
ChildrenGovernment

Harm types
PsychologicalPublic interest

AI system task:
Content generation

In other databases


Articles about this incident or hazard