Facebook Deploys AI Tools to Combat Child Exploitation Content

Thumbnail Image

The information displayed in the AIM should not be reported as representing the official views of the OECD or of its member countries.

Facebook has implemented and tested new AI-driven tools, including Google's Content Safety API and behavioral detection systems, to identify, remove, and report child exploitative content on its platforms. These measures aim to prevent harm to children by detecting both explicit and contextual sexualization, alerting users, and disabling offending accounts.[AI generated]

Why's our monitor labelling this an incident or hazard?

The event describes the use of AI systems to detect and moderate child exploitation content on Facebook and Instagram. The AI tools analyze both images and contextual text to identify sexualization of children, which is a violation of human rights and causes harm to communities. The deployment of these tools and removal of harmful content directly mitigates ongoing harm, qualifying this as an AI Incident under the framework because the AI system's use has directly led to harm reduction related to child exploitation.[AI generated]
Industries
Media, social platforms, and marketingDigital security

Business function:
Monitoring and quality control

AI system task:
Recognition/object detectionEvent/anomaly detection


Articles about this incident or hazard