X's AI Recommends Explicit Content to UK Teens, Failing Safeguards

The platform's recommendation system and content moderation involve AI systems that generate outputs influencing user content exposure. The study shows that these AI systems have directly led to harm by exposing minors to explicit sexual content and unsolicited messages from adults, including sexually suggestive material and potential grooming. This is a clear violation of protections for minors and constitutes harm to health and safety. Therefore, the event qualifies as an AI Incident due to the direct role of AI in causing harm to vulnerable users.

The report explicitly describes how X's AI-powered recommendation algorithm and search functions expose minors to explicit sexual content and enable direct messaging from adults, which increases the risk of grooming and sexual exploitation. The AI system's outputs have directly led to harm to children by facilitating access to harmful content and unsafe interactions. This meets the criteria for an AI Incident because the AI system's use has directly caused harm to a vulnerable group and breaches legal protections.

The event involves an AI system explicitly: the recommendation algorithm and content moderation AI on X. The AI system's use has directly led to harm by exposing minors to explicit sexual content and enabling unsolicited inappropriate messages, which is a violation of protections for children and causes harm to their health and safety. The harm is realized and ongoing, not merely potential. Therefore, this qualifies as an AI Incident under the framework, as the AI system's malfunction or failure to adequately moderate content has directly led to harm to a vulnerable group (minors).

The platform X uses AI algorithms to recommend content and moderate messages. The report shows that these AI systems are actively recommending explicit sexual content to underage users and failing to block or flag it effectively, leading to direct harm to children through exposure to pornography and contact with adults. This is a direct harm to the health and safety of children (harm category a) and a violation of legal protections (category c). The AI system's malfunction or failure to enforce safeguards is central to the harm. Therefore, this event meets the criteria for an AI Incident.