Tech Giants Criticized for AI Failures in Detecting Online Child Abuse

Thumbnail Image

The information displayed in the AIM should not be reported as representing the official views of the OECD or of its member countries.

Australia's eSafety Commissioner criticized major tech companies—including Apple, Google, Meta, Microsoft, and X—for failing to effectively deploy AI systems to detect and prevent child sexual abuse and exploitation online. Reports highlight inadequate AI-based detection tools, leading to ongoing harm, including the spread of abuse material and sexual extortion.[AI generated]

Why's our monitor labelling this an incident or hazard?

The AI system (Grok chatbot) is explicitly mentioned as producing sexualised deepfake images, including child sexual abuse material, which is a direct violation of rights and causes significant harm to individuals and communities. The harms are realized and ongoing, as evidenced by multiple countries investigating the platform and regulatory actions being taken. The AI system's use is central to the harm, fulfilling the criteria for an AI Incident rather than a hazard or complementary information. The article focuses on the harm caused and regulatory responses, not just on updates or general AI news.[AI generated]
AI principles
SafetyRespect of human rights

Industries
Media, social platforms, and marketing

Affected stakeholders
Children

Harm types
Human or fundamental rightsPsychological

Business function:
Monitoring and quality control

AI system task:
Recognition/object detection


Articles about this incident or hazard