School Photo Posts Vulnerable To Emerging AI Threat
2026-07-27
Mirage News
The information displayed in the AIM should not be reported as representing the official views of the OECD or of its member countries.
Australian schools have seen a surge in AI-generated deepfake content using images from school social media accounts. The eSafety Commissioner reported over 100 cases of malicious AI-created videos, memes, and child sexual exploitation material, causing reputational harm and distress to students and staff. Schools are urged to reconsider posting student photos online.[AI generated]
Why's our monitor labelling this an incident or hazard?
The event involves AI systems used maliciously to generate harmful content targeting school communities, resulting in realized harm such as reputational damage, distress, and violations of rights. The AI systems' use in creating deepfake child sexual exploitation material and other fabricated content constitutes direct harm to individuals and communities. Therefore, this qualifies as an AI Incident. The article also discusses governance responses, but the primary focus is on the realized harms caused by AI misuse.[AI generated]