AI-Generated Sex Worker Ads Flood Social Media Platforms

Thumbnail Image

The information displayed in the AIM should not be reported as representing the official views of the OECD or of its member countries.

AI-powered chatbots and virtual companions offering sexually explicit content are being widely advertised on Instagram, Facebook, and TikTok, bypassing content moderation systems. These ads, often targeting men and sometimes using children's characters, have led to the spread of harmful, exploitative, and inappropriate AI-generated content, violating platform policies.[AI generated]

Why's our monitor labelling this an incident or hazard?

The event involves AI systems explicitly described as using machine learning to generate sexualized virtual companions and content. The use of these AI systems has led to the dissemination of sexually explicit ads that evade moderation, potentially exposing users to harmful content and violating platform policies. This constitutes indirect harm to communities and possible violations of rights (e.g., exploitation, exposure to inappropriate content). Therefore, this qualifies as an AI Incident because the AI system's use has directly led to harm through the spread of inappropriate and potentially exploitative content on social media platforms.[AI generated]
AI principles
SafetyRespect of human rightsRobustness & digital securityAccountabilityTransparency & explainabilityHuman wellbeing
Industries
Media, social platforms, and marketingDigital securityConsumer services
Affected stakeholders
ChildrenGeneral publicBusinessConsumers
Harm types
Human or fundamental rightsPsychologicalReputational
Business function:
Marketing and advertisementCitizen/customer serviceICT management and information security
AI system task:
Interaction support/chatbotsContent generationOrganisation/recommenders

Articles about this incident or hazard