The information displayed in the AIM should not be reported as representing the official views of the OECD or of its member countries.
Over 400 AI-powered apps, disguised as harmless tools like pet photo editors and language tutors, secretly enabled users to create non-consensual sexually explicit and violent deepfake images. Operated by a Chinese-linked network, these apps bypassed Apple and Google vetting, causing significant harm before removal and ad bans by major tech platforms.[AI generated]
Why's our monitor labelling this an incident or hazard?
The event involves AI systems that generate sexually abusive deepfake content, including CSAM, which is a severe violation of human rights and legal protections. The AI's use in producing and distributing such harmful content directly leads to harm to individuals and communities. The network's deceptive practices and scale amplify the impact. The presence of AI is explicit in generating deepfake images and videos, and the harm is realized, not just potential. Therefore, this qualifies as an AI Incident under the framework, specifically under violations of human rights and harm to communities.[AI generated]