AI Medical Tools Prone to Authoritative Misinformation, Study Finds

Thumbnail Image

The information displayed in the AIM should not be reported as representing the official views of the OECD or of its member countries.

A study published in The Lancet Digital Health found that 20 large language models, including medical AI tools, are more likely to accept and propagate false medical information when it appears in authoritative formats like doctors' notes, posing risks of incorrect medical advice and potential harm to patient health.[AI generated]

Why's our monitor labelling this an incident or hazard?

The article involves AI systems (large language models) and their use in medical contexts, showing that their outputs can propagate false medical information. This poses a plausible risk of harm to health (harm category a) if such misinformation is acted upon. However, the article does not describe an actual incident where harm has occurred, but rather a study highlighting vulnerabilities and potential for harm. Therefore, it fits the definition of an AI Hazard, as it plausibly could lead to an AI Incident involving harm to health if unaddressed.[AI generated]
AI principles
SafetyRobustness & digital security

Industries
Healthcare, drugs, and biotechnology

Affected stakeholders
Consumers

Harm types
Physical (injury)

AI system task:
Content generation


Articles about this incident or hazard