Indonesia Blocks Grok AI Over Deepfake Sexual Content Concerns

Thumbnail Image

The information displayed in the AIM should not be reported as representing the official views of the OECD or of its member countries.

Indonesian authorities have temporarily blocked Grok AI, developed by xAI and used on the X platform, due to its misuse in generating deepfake sexual content. The government warns of a potential permanent ban if the platform fails to comply with national regulations aimed at preventing AI-driven harm.[AI generated]

Why's our monitor labelling this an incident or hazard?

Grok is an AI chatbot that generates content, including images. The generation and spread of obscene images without consent constitutes harm to individuals' rights and communities, fitting the definition of an AI Incident. The involvement of the AI system's outputs directly led to regulatory blocking due to violations and harm. Therefore, this event qualifies as an AI Incident rather than a hazard or complementary information.[AI generated]
AI principles
SafetyRespect of human rights

Industries
Media, social platforms, and marketingDigital security

Affected stakeholders
General public

Harm types
ReputationalPsychologicalHuman or fundamental rights

Severity
AI incident

AI system task:
Content generation


Articles about this incident or hazard

Thumbnail Image

Komdigi Buka Peluang Blokir Permanen Grok jika Tak Patuhi Aturan di Indonesia

2026-01-27
KOMPAS.com
Why's our monitor labelling this an incident or hazard?
The article explicitly mentions Grok as an AI system and discusses regulatory measures and potential blocking if rules are not followed. There is no indication that Grok has caused any direct or indirect harm yet, only that authorities are preparing enforcement actions to ensure compliance and protect public safety. This fits the definition of Complementary Information, which includes governance responses and updates related to AI systems without reporting a new incident or hazard.
Thumbnail Image

RI Siap Blokir Permanen Grok Buatan Elon Musk

2026-01-28
CNBCindonesia
Why's our monitor labelling this an incident or hazard?
Grok is an AI chatbot that generates content, including images. The generation and spread of obscene images without consent constitutes harm to individuals' rights and communities, fitting the definition of an AI Incident. The involvement of the AI system's outputs directly led to regulatory blocking due to violations and harm. Therefore, this event qualifies as an AI Incident rather than a hazard or complementary information.
Thumbnail Image

Komdigi Berpotensi Blokir Permanen Grok AI Milik Elon Musk

2026-01-27
detiki net
Why's our monitor labelling this an incident or hazard?
The article explicitly mentions an AI system (Grok AI) and its misuse potential (manipulating photos into indecent images), which is a form of harm related to violations of rights and ethical standards. The government has blocked the service temporarily and is considering permanent blocking if compliance is not met. Since the harm is related to potential misuse and regulatory non-compliance rather than a confirmed widespread or direct harm incident, this fits the definition of an AI Hazard. The event involves the use of an AI system and plausible future harm if the system continues to operate without compliance, but no confirmed direct or indirect harm has been reported yet.
Thumbnail Image

Video: Grok AI Masih Diblokir, Pemerintah Tunggu X Patuhi Aturan Indonesia

2026-01-28
20DETIK
Why's our monitor labelling this an incident or hazard?
The article involves an AI system (Grok AI) and discusses regulatory compliance and potential misuse leading to harmful content (e.g., obscene content). Since no actual harm or incident is reported but there is a credible risk of harm if misuse continues, this situation qualifies as an AI Hazard. The government's monitoring and potential blocking represent a response to plausible future harm rather than a realized incident.
Thumbnail Image

Komdigi Bakal Blokir Grok AI secara Permanen Jika Tak Patuhi Regulasi

2026-01-27
VOI - Waktunya Merevolusi Pemberitaan
Why's our monitor labelling this an incident or hazard?
Grok AI is an AI system used on the X platform, and its misuse to create deepfake sexual content is a direct cause of harm to communities and a violation of rights. The article reports on the government's regulatory actions and warnings of permanent blocking if compliance is not met. Since harm from misuse is already occurring (production and spread of deepfake pornography), this qualifies as an AI Incident. The regulatory response and potential blocking are complementary to the incident but do not overshadow the fact that harm is realized due to AI misuse.