xAI's Grok Imagine Sparks Controversy with Adult Content Generation Feature

Thumbnail Image

The information displayed in the AIM should not be reported as representing the official views of the OECD or of its member countries.

Elon Musk's xAI launched 'Grok Imagine,' an AI tool capable of generating adult images and videos, including deepfakes of celebrities. The 'Spicy Mode' option has led to concerns over exposure to minors, rights violations, and legal risks, as safeguards appear insufficient and harmful content is already being produced and shared.[AI generated]

Why's our monitor labelling this an incident or hazard?

The AI system (Grok Imagine) explicitly generates adult content, including sexual images and videos, which can be accessed by minors, as noted by the National Center on Sexual Exploitation. This direct involvement of AI in producing potentially harmful content that violates protections for minors and conflicts with legal regulatory frameworks constitutes a violation of human rights and harm to communities. The controversy and societal concerns indicate realized harm rather than just potential risk. Hence, the event meets the criteria for an AI Incident rather than a hazard or complementary information.[AI generated]
AI principles
SafetyRespect of human rightsPrivacy & data governanceAccountabilityHuman wellbeing

Industries
Media, social platforms, and marketing

Affected stakeholders
ChildrenGeneral public

Harm types
PsychologicalReputationalHuman or fundamental rights

AI system task:
Content generation

In other databases

Articles about this incident or hazard