EU and Dutch Court Ban AI-Generated Sexual Deepfakes After Harmful Incidents

Thumbnail Image

The information displayed in the AIM should not be reported as representing the official views of the OECD or of its member countries.

The European Parliament and a Dutch court have banned AI systems, including Grok on X, from generating non-consensual sexual deepfakes and child exploitation content. These actions follow incidents where AI-generated images caused privacy violations and psychological harm, prompting regulatory measures to protect individuals, especially women and children, in the EU.[AI generated]

Why's our monitor labelling this an incident or hazard?

The presence of an AI system (Grok) explicitly involved in generating harmful sexual deepfakes is clear. The misuse of this AI system has directly led to violations of individuals' rights and harm to their dignity, fitting the definition of an AI Incident. The European Parliament's vote to ban such AI systems is a governance response to this incident. Since the article focuses on the scandal and the legislative action taken in response to realized harm, the classification is AI Incident. The mention of regulatory delays and investigations adds complementary context but does not change the primary classification.[AI generated]
AI principles
Privacy & data governanceRespect of human rights

Industries
Media, social platforms, and marketing

Affected stakeholders
WomenChildren

Harm types
PsychologicalHuman or fundamental rights

AI system task:
Content generation


Articles about this incident or hazard