India Probes Meta Over AI Failure Allowing Child Abuse Ads on Instagram

Thumbnail Image

The information displayed in the AIM should not be reported as representing the official views of the OECD or of its member countries.

India's National Commission for Protection of Child Rights (NCPCR) has launched a formal inquiry into Meta after AI-driven content moderation and ad systems on Instagram failed to block advertisements linked to child sexual exploitation. The incident follows a BBC investigation and highlights AI system shortcomings in preventing harmful content.[AI generated]

Why's our monitor labelling this an incident or hazard?

Instagram's content moderation system likely uses AI to detect and block harmful content before publication. The failure of this system to prevent child pornography ads from going live constitutes a malfunction of an AI system that directly led to harm by enabling access to illegal and harmful material. This meets the criteria for an AI Incident due to violations of human rights and harm to communities caused by the AI system's failure.[AI generated]
AI principles
Respect of human rightsSafety

Industries
Media, social platforms, and marketingDigital security

Affected stakeholders
Children

Harm types
Human or fundamental rightsPsychological

Business function:
Marketing and advertisement

AI system task:
Recognition/object detectionOrganisation/recommenders


Articles about this incident or hazard