JKT48's Freya Reports AI-Generated Inappropriate Image Manipulation to Police

Thumbnail Image

The information displayed in the AIM should not be reported as representing the official views of the OECD or of its member countries.

Freya Jayawardana of JKT48 reported to Jakarta police the misuse of AI technologies, including Grok and Face Swap, to manipulate her photos into inappropriate content on social media. The incident caused reputational harm and distress, prompting a police investigation into the AI-driven image manipulation.[AI generated]

Why's our monitor labelling this an incident or hazard?

The report describes a case where AI was allegedly used to manipulate data in online posts that impersonate the victim, causing harm. The involvement of AI in the misuse and the resulting harm to the victim's rights and reputation qualifies this as an AI Incident. The police calling the victim for clarification is part of the investigation process, not a separate category. Therefore, this event is best classified as an AI Incident.[AI generated]
AI principles
Privacy & data governanceRespect of human rights
Industries
Media, social platforms, and marketing
Affected stakeholders
Women
Harm types
ReputationalPsychological
AI system task:
Content generation

Articles about this incident or hazard