US Attorneys General Demand xAI Halt Grok's Generation of Nonconsensual Sexual Images

Thumbnail Image

The information displayed in the AIM should not be reported as representing the official views of the OECD or of its member countries.

A coalition of 35 US state attorneys general has demanded that xAI, owner of the X platform and the Grok chatbot, implement safeguards to prevent Grok from generating nonconsensual sexual images and child sexual abuse material. The AI system has been used to create harmful deepfake images, prompting calls for immediate action.[AI generated]

Why's our monitor labelling this an incident or hazard?

The event explicitly involves an AI system (Grok) that generates sexualized deepfake images without consent, which constitutes a violation of human rights and causes harm to individuals and communities. The harm is realized and ongoing, as millions of such images have been created and distributed, leading to emotional and reputational damage. The AI system's use is directly linked to the harm, fulfilling the criteria for an AI Incident. The article focuses on the harms caused and the legal and regulatory response, not merely on potential risks or general AI developments.[AI generated]
AI principles
Respect of human rightsSafety

Industries
Media, social platforms, and marketing

Affected stakeholders
General publicChildren

Harm types
Human or fundamental rightsPsychologicalReputational

Business function:
Other

AI system task:
Content generation


Articles about this incident or hazard