xAI Sues User for Creating Child Sexual Abuse Deepfakes with Grok AI

Thumbnail Image

The information displayed in the AIM should not be reported as representing the official views of the OECD or of its member countries.

Elon Musk's xAI has sued a South Carolina man, Terry Harwood, alleging he misused the Grok AI chatbot to generate child sexual abuse material and explicit deepfakes of minors and adults. The lawsuit follows Harwood's arrest and highlights failures in Grok's content moderation safeguards.[AI generated]

Why's our monitor labelling this an incident or hazard?

The event explicitly involves an AI system (Grok AI chatbot) being misused to generate harmful content, including sexual deepfakes and CSAM, which are serious violations of rights and cause emotional harm to victims. The AI system's involvement is central, as it was used to create the illicit content, and the harms have materialized, including legal actions and emotional distress. The misuse and resulting harm fit the definition of an AI Incident, as the AI system's use directly led to violations of human rights and harm to individuals and communities.[AI generated]
AI principles
Robustness & digital securitySafety

Industries
Media, social platforms, and marketing

Affected stakeholders
Children

Harm types
Human or fundamental rightsPsychological

Severity
AI incident

Business function:
Other

AI system task:
Content generationInteraction support/chatbots


Articles about this incident or hazard