
The information displayed in the AIM should not be reported as representing the official views of the OECD or of its member countries.
The social media app Giggle uses facial recognition AI to determine user gender, resulting in the exclusion of transgender women and misidentification of women of color. This AI-driven gatekeeping has led to discrimination and denial of access, sparking criticism for violating the rights of marginalized groups.[AI generated]
Why's our monitor labelling this an incident or hazard?
The facial recognition AI is explicitly used to gatekeep access to the app based on gender classification, which has led to the exclusion of transgender women and misidentification of women of color. This constitutes a violation of rights and harm to communities, fulfilling the criteria for an AI Incident. The harm is realized and ongoing, not merely potential. The AI system's role is pivotal in causing these harms, as it is the mechanism enforcing exclusion and discrimination.[AI generated]