The information displayed in the AIM should not be reported as representing the official views of the OECD or of its member countries.
Google's Med-Gemini healthcare AI generated a hallucinated term, 'basilar ganglia,' conflating two distinct brain structures in a published research paper. The error, initially unnoticed by authors and reviewers, highlights risks of AI-generated medical misinformation and the potential for harm if such mistakes go undetected in clinical settings.[AI generated]
Why's our monitor labelling this an incident or hazard?
The event involves an AI system explicitly named (Google's Med-Gemini) used in healthcare for interpreting medical scans and generating reports. The AI system produced a hallucinated term "basilar ganglia," which is medically incorrect and could lead to misdiagnosis and inappropriate treatment, posing a direct risk of harm to patients' health. The article documents that this error was not caught initially and remains in the research paper, indicating a malfunction in the AI system's output. Medical experts express concern about the dangers of such hallucinations and the risk of automation bias leading to missed errors by clinicians relying on AI. The harm is related to injury or harm to health (definition a), and the AI system's malfunction is a contributing factor. Hence, this event meets the criteria for an AI Incident rather than a hazard or complementary information.[AI generated]
