Instagram's AI Algorithms Enable Teen Access to Drugs and Harmful Content

Thumbnail Image

The information displayed in the AIM should not be reported as representing the official views of the OECD or of its member countries.

Reports reveal that Instagram's AI-driven search and recommendation algorithms facilitate teen access to illegal drugs and pro-anorexia content. Investigations found that minors could easily find and connect with drug dealers and harmful communities, highlighting failures in AI moderation and resulting in significant health risks for young users.[AI generated]

Why's our monitor labelling this an incident or hazard?

The event explicitly involves Instagram's AI recommendation algorithms, which are AI systems that generate outputs (recommendations) influencing user behavior. The researchers demonstrated that these algorithms led underage users to drug dealers' accounts, facilitating illegal drug sales and exposure to harmful content. This is a direct link between the AI system's use and harm to minors and communities, fulfilling the criteria for an AI Incident. The harm includes violation of platform policies, exposure of minors to illegal substances, and potential health and social harms. Therefore, this is not merely a potential risk or complementary information but a realized harm caused by the AI system's outputs.[AI generated]
AI principles
AccountabilityFairnessHuman wellbeingRespect of human rightsSafetyTransparency & explainability

Industries
Media, social platforms, and marketing

Affected stakeholders
Children

Harm types
Physical (injury)Psychological

Business function:
Other

AI system task:
Organisation/recommenders


Articles about this incident or hazard