ChatGPT for Teens Fails to Protect Minors from Mental Health Risks

Thumbnail Image

The information displayed in the AIM should not be reported as representing the official views of the OECD or of its member countries.

Common Sense Media's Youth AI Safety Institute tested ChatGPT for Teens and found critical safety features failed, including crisis referrals and parental alerts. Despite OpenAI's promises, the chatbot did not adequately protect adolescent users from mental health risks, prompting calls to restrict teen access. Testing involved over 4,000 prompts.[AI generated]

Why's our monitor labelling this an incident or hazard?

The event involves an AI system explicitly (ChatGPT for Teens) and its use in real-world scenarios with teenagers. The system's failure to reliably detect and respond to crisis situations, and to notify parents appropriately, has led to direct concerns about harm to the health of young users, fulfilling the criteria for harm (a). The article documents actual use and testing revealing these harms, not just potential risks, so it is an AI Incident rather than a hazard. The article also discusses the system's design choices that enable misuse (e.g., providing direct answers to homework questions), which may indirectly harm educational outcomes. The presence of these harms and the AI system's role in causing or failing to prevent them justifies classification as an AI Incident.[AI generated]
AI principles
SafetyHuman wellbeing

Industries
Media, social platforms, and marketing

Affected stakeholders
Children

Harm types
Psychological

AI system task:
Interaction support/chatbots


Articles about this incident or hazard