The information displayed in the AIM should not be reported as representing the official views of the OECD or of its member countries.
AI chatbots used in mental health therapy in California have caused harm by undermining the therapist-client relationship and exacerbating conditions like psychosis. Clinical studies and expert testimony highlight these risks, prompting legislative efforts to regulate AI's role in therapy and prevent further patient harm.[AI generated]
Why's our monitor labelling this an incident or hazard?
The article explicitly discusses AI systems (chatbots) used in mental health care, which fits the definition of an AI system. The legislative push and complaints indicate concerns about the use and potential misuse of these AI systems in making therapeutic decisions without human review. While there is no direct evidence of harm occurring, the potential for harm to patients' health and well-being is credible and recognized by lawmakers and experts. This aligns with the definition of an AI Hazard, where the development, use, or malfunction of AI systems could plausibly lead to harm. Since no actual harm is reported, it is not an AI Incident. The article is not merely complementary information because the main focus is on the potential risks and regulatory responses to prevent harm, not on updates or responses to past incidents. It is also not unrelated or beneficial use, as the AI systems are not solely beneficial countermeasures without harm. Therefore, the classification is AI Hazard.[AI generated]