Malaysia Deepfake Incident Spurs Calls for Stricter School Digital Safety

Thumbnail Image

The information displayed in the AIM should not be reported as representing the official views of the OECD or of its member countries.

Deputy Communications Minister Teo Nie Ching has called for urgent digital safety reforms in Malaysian schools after AI-generated explicit images of 38 students, some as young as 12, were circulated. The incident highlights misuse of deepfake technology, breaches human rights, and necessitates stricter protocols in educational institutions.[AI generated]

Why's our monitor labelling this an incident or hazard?

The event involves the use of AI systems (deepfake technology) to produce harmful content that directly harms individuals (students) by violating their rights and causing psychological and social harm. The harm has already occurred as victims have been identified and complaints filed. Therefore, this qualifies as an AI Incident due to realized harm caused by AI-generated content. The focus on the need for stricter protocols is a response to the incident but does not change the classification.[AI generated]
AI principles
Respect of human rightsPrivacy & data governanceSafetyHuman wellbeingRobustness & digital securityAccountabilityTransparency & explainability

Industries
Education and trainingMedia, social platforms, and marketingDigital securityGovernment, security, and defence

Affected stakeholders
Children

Harm types
Human or fundamental rightsPsychologicalReputationalPublic interest

AI system task:
Content generation


Articles about this incident or hazard