Kota Police and Meta Use AI Monitoring to Prevent Student Suicides

Thumbnail Image

The information displayed in the AIM should not be reported as representing the official views of the OECD or of its member countries.

Kota police partnered with Meta to use AI-powered monitoring of Facebook and Instagram to detect suicidal content among coaching students. Within a week, officers intervened to avert a suicide attempt from a student in Jhunjhunu. The initiative aims to scale statewide detection and response to prevent further student suicides.[AI generated]

Why's our monitor labelling this an incident or hazard?

Meta's social media platforms likely use AI systems to analyze user behavior and content to detect signs of suicidal tendencies. The collaboration with the police to intervene and prevent suicides indicates that the AI system's use has directly led to harm prevention, specifically injury or harm to health. Therefore, this qualifies as an AI Incident because the AI system's use has directly led to harm prevention, which is a positive form of harm management under the framework.[AI generated]
AI principles
Privacy & data governanceTransparency & explainabilityRespect of human rights
Industries
Media, social platforms, and marketingGovernment, security, and defence
Affected stakeholders
General public
Business function:
Compliance and justice
AI system task:
Event/anomaly detection

Articles about this incident or hazard