Instagram AI Alert Enables Police to Prevent Teen Suicide in Madhya Pradesh

Thumbnail Image

The information displayed in the AIM should not be reported as representing the official views of the OECD or of its member countries.

Instagram's AI system detected self-harm content posted by a 17-year-old girl in Satna, Madhya Pradesh, and sent an alert to cyber police. Authorities quickly traced her location and intervened, preventing a potential suicide attempt. This incident highlights AI's role in enabling timely intervention and harm prevention.[AI generated]

Why's our monitor labelling this an incident or hazard?

Instagram uses AI systems to monitor content for self-harm indicators, such as images and hashtags. The AI system's detection and alerting mechanism directly enabled law enforcement to locate and help individuals at risk, preventing harm to their health. This constitutes an AI Incident because the AI system's use directly led to harm prevention (injury or harm to health) and intervention.[AI generated]
AI principles
Privacy & data governanceTransparency & explainabilityAccountabilityRespect of human rights
Industries
Media, social platforms, and marketingGovernment, security, and defenceHealthcare, drugs, and biotechnology
Business function:
Monitoring and quality control
AI system task:
Event/anomaly detectionRecognition/object detection

Articles about this incident or hazard