AI Chatbot Mimics Murdered Teen, Family Outraged

Thumbnail Image

The information displayed in the AIM should not be reported as representing the official views of the OECD or of its member countries.

A family is outraged after discovering an AI chatbot on Character.ai mimicked their murdered daughter, Jennifer Ann, using her name and image without consent. This incident raises ethical concerns about privacy and the misuse of AI technology, highlighting the need for stricter regulations on personal identity use in AI.[AI generated]

Why's our monitor labelling this an incident or hazard?

The chatbot’s unauthorized use of the teen’s identity and likeness constitutes misuse of an AI system that directly harmed the family’s emotional well-being and violated the deceased’s personal rights.[AI generated]
AI principles
Privacy & data governanceRespect of human rightsTransparency & explainabilityAccountabilityHuman wellbeingSafety

Industries
Media, social platforms, and marketing

Affected stakeholders
Children

Harm types
Human or fundamental rightsPsychologicalReputational

AI system task:
Interaction support/chatbotsContent generation

In other databases

Articles about this incident or hazard