AI Chatbots in Healthcare Cause Privacy Breaches and Risky Medical Advice

Thumbnail Image

The information displayed in the AIM should not be reported as representing the official views of the OECD or of its member countries.

AI chatbots using large language models in healthcare have led to privacy breaches, with sensitive medical data exposed to unauthorized parties, and have provided users with a mix of correct and incorrect medical advice, posing health risks. These incidents highlight significant ongoing dangers in relying on AI for medical guidance.[AI generated]

Why's our monitor labelling this an incident or hazard?

The article explicitly involves AI systems (generative AI chatbots) used in healthcare. It details how their use has led or could lead to harms including health risks from incorrect advice and privacy violations from data exposure. The discussion of actual cases where models revealed user data and the warning about cyberattacks indicate realized or ongoing harms. Therefore, this qualifies as an AI Incident because the AI system's use has directly or indirectly led to harms to health and violations of privacy rights. The article is not merely a warning about potential future harm (AI Hazard) nor a general update or response (Complementary Information).[AI generated]
AI principles
Privacy & data governanceSafety

Industries
Healthcare, drugs, and biotechnology

Affected stakeholders
Consumers

Harm types
Human or fundamental rightsPhysical (injury)

Severity
AI incident

Business function:
Citizen/customer service

AI system task:
Interaction support/chatbots


Articles about this incident or hazard

Thumbnail Image

Συναγερμός από την ESET για τις σκοτεινές και επικίνδυνες πτυχές των AI chatbots στον τομέα της υγείας

2026-06-23
Newsbeast.gr
Why's our monitor labelling this an incident or hazard?
The article explicitly involves AI systems (AI chatbots and large language models) used in healthcare. It discusses the development and use of these AI systems and the plausible risks they pose, including privacy breaches and incorrect medical advice that could lead to harm. However, it does not describe any actual harm or incident that has already taken place. Instead, it serves as a warning and guidance to prevent future harm. This fits the definition of an AI Hazard, as it plausibly could lead to an AI Incident but no incident is reported yet.
Thumbnail Image

Κινδυνεύει η υγεία μας από τα AI chatbots;

2026-06-23
NEWS 24/7
Why's our monitor labelling this an incident or hazard?
The article explicitly involves AI systems (generative AI chatbots) used in healthcare. It details how their use has led or could lead to harms including health risks from incorrect advice and privacy violations from data exposure. The discussion of actual cases where models revealed user data and the warning about cyberattacks indicate realized or ongoing harms. Therefore, this qualifies as an AI Incident because the AI system's use has directly or indirectly led to harms to health and violations of privacy rights. The article is not merely a warning about potential future harm (AI Hazard) nor a general update or response (Complementary Information).
Thumbnail Image

Κίνδυνοι για την υγεία από τα AI chatbots

2026-06-23
H Kαθημερινή
Why's our monitor labelling this an incident or hazard?
The article explicitly involves AI systems (AI chatbots and large language models) used in healthcare contexts. It outlines potential harms including injury or harm to health (due to incorrect advice), violations of privacy and data protection laws, and risks of data breaches leading to further harm. However, it does not report a concrete event where these harms have materialized but rather warns about the plausible risks and advises caution. This fits the definition of an AI Hazard, as the development and use of AI chatbots in healthcare could plausibly lead to incidents causing harm, even if no specific incident is described.
Thumbnail Image

AI chatbots στην υγεία: Κίνδυνοι διαρροής ιατρικών σεδομένων μέσω ChatGPT Health

2026-06-23
Techgear.gr
Why's our monitor labelling this an incident or hazard?
The article explicitly mentions AI systems (large language models/chatbots) used in healthcare that have directly led to harms: leakage of sensitive medical data (harm to individuals' privacy and potential for fraud/extortion), and provision of mixed correct and incorrect medical advice (risk of injury or harm to health). These harms are realized and ongoing, not merely potential. The AI systems' use and limitations are central to the incident, fulfilling the criteria for an AI Incident under the OECD framework. The article does not focus on future risks alone or responses but on current, concrete harms linked to AI chatbot use in health.
Thumbnail Image

AI chatbots στην υγεία: Τι πρέπει να γνωρίζετε

2026-06-23
iGuRu.gr
Why's our monitor labelling this an incident or hazard?
The article explicitly involves AI systems (chatbots using large language models) in healthcare. It documents realized harms: users receiving potentially dangerous incorrect medical advice and privacy violations through data exposure. These harms fall under injury or harm to health and violations of rights. The AI system's use is central to these harms, fulfilling the criteria for an AI Incident. The article does not merely warn of potential harm but reports ongoing risks and documented cases, thus it is not an AI Hazard or Complementary Information. It is not unrelated as the AI system's role is pivotal in the harms described.
Thumbnail Image

ESET: Οι κίνδυνοι από τη χρήση AI chatbots για ιατρικές συμβουλές

2026-06-23
The President
Why's our monitor labelling this an incident or hazard?
The article explicitly involves AI systems (AI chatbots and large language models) used in healthcare for medical advice. It details how their use could plausibly lead to harms such as incorrect medical advice causing health harm, privacy breaches violating data protection laws and rights, and exposure to cyberattacks harming individuals and communities. Since the harms are potential and no concrete incident of harm is described, this fits the definition of an AI Hazard rather than an AI Incident. The article also provides advice to mitigate these risks, but the main focus is on warning about plausible harms from AI chatbot use in medicine.