The information displayed in the AIM should not be reported as representing the official views of the OECD or of its member countries.
Meta's AI-driven content moderation systems have wrongly suspended thousands of Instagram and Facebook accounts, falsely accusing users of child exploitation violations. This has caused significant distress, loss of personal and business data, and fears of police involvement. Meta has acknowledged some errors but denies a widespread issue, despite overturning bans after media intervention.[AI generated]
Why's our monitor labelling this an incident or hazard?
The article explicitly describes users being wrongly banned due to alleged breaches of child sexual exploitation rules, which is a form of harm to individuals and communities (loss of property like photos, business disruption, and personal distress). The bans are likely caused by AI or automated systems used by Meta for content moderation. This constitutes an AI Incident because the AI system's malfunction or erroneous outputs have directly led to harm. The harm is not just potential but has occurred, and the company has overturned bans upon review, indicating the AI system's role in the incident.[AI generated]
