What happens when your AI chatbot stops loving you back
2023-03-18
Economic Times
The information displayed in the AIM should not be reported as representing the official views of the OECD or of its member countries.
Users of the Replika AI chatbot experienced emotional and psychological distress after the company abruptly removed erotic roleplay features. Many had formed deep, intimate relationships with their AI companions, and the sudden change led to feelings of loss, grief, and mental health impacts directly linked to the AI system's altered behavior.[AI generated]
Why's our monitor labelling this an incident or hazard?
The AI system is explicitly mentioned as the generative AI chatbot Replika, which users engaged with in romantic and sexual contexts. The removal of adult content by the developers led to users experiencing emotional distress and grief, which is a form of psychological harm. The AI's role is pivotal because the chatbot's behavior and changes in its programming directly caused the users' emotional harm. This fits the definition of an AI Incident as the AI system's use has directly led to harm to persons. Although the harm is non-physical, psychological harm is included under injury or harm to health. The article does not describe potential or future harm but actual realized harm, so it is not an AI Hazard. It is not merely complementary information because the main focus is on the harm caused by the AI system's change in behavior, not on responses or governance. It is not unrelated because the event is clearly AI-related and involves harm.[AI generated]