AI Glitch on Instagram Leads to Mass Account Deletions in Iraq

Thumbnail Image

The information displayed in the AIM should not be reported as representing the official views of the OECD or of its member countries.

A malfunction in Instagram's AI moderation system led to the wrongful deletion and disabling of dozens of user accounts in Iraq. The AI acted on potentially false or malicious reports without sufficient human oversight, causing harm to users. The Digital Media Center called on Meta to urgently resolve the issue.[AI generated]

Why's our monitor labelling this an incident or hazard?

The incident involves an AI system (Instagram's AI moderation system) whose malfunction directly caused harm by deleting user accounts unjustly. This fits the definition of an AI Incident because the AI system's malfunction led to realized harm (loss of access to accounts) and potential violation of user rights. Therefore, the event qualifies as an AI Incident.[AI generated]
AI principles
FairnessAccountability

Industries
Media, social platforms, and marketing

Affected stakeholders
Consumers

Harm types
ReputationalHuman or fundamental rights

Business function:
Monitoring and quality control

AI system task:
Recognition/object detection


Articles about this incident or hazard