AI Chatbot Medical Advice Causes Hospitalization Due to Poisoning

Thumbnail Image

The information displayed in the AIM should not be reported as representing the official views of the OECD or of its member countries.

Researchers from several US universities warn that AI chatbots, like ChatGPT, can provide dangerous medical advice. A documented case involved a 60-year-old man hospitalized for bromide poisoning after following AI advice to replace table salt with sodium bromide. The incident highlights risks and lack of oversight in AI-generated health information.[AI generated]

Why's our monitor labelling this an incident or hazard?

The presence of AI systems is explicit (ChatGPT and AI models used in social media content processing). The harm is direct and realized: a user suffered poisoning due to following AI medical advice, and the inability of doctors to access the AI advice delayed treatment, worsening the harm. The article also discusses broader risks of AI-generated medical misinformation affecting users' health. These constitute injury or harm to health (criterion a). Hence, the event is an AI Incident. The article also discusses proposed solutions and governance responses, but the main focus is on the harm caused by AI use, not just responses, so it is not Complementary Information.[AI generated]
AI principles
SafetyAccountability

Industries
Healthcare, drugs, and biotechnology

Affected stakeholders
Consumers

Harm types
Physical (injury)

AI system task:
Interaction support/chatbots


Articles about this incident or hazard