AI Scribes in NHS Misrecord Diagnoses and Medications, Causing Patient Harm

Thumbnail Image

The information displayed in the AIM should not be reported as representing the official views of the OECD or of its member countries.

AI 'ambient scribe' tools used in NHS England have incorrectly transcribed medical consultations, inventing diagnoses like multiple sclerosis and confusing drug names. Healthwatch England documented cases where these errors led to patient distress and potential health risks, raising concerns about patient safety and regulatory oversight.[AI generated]

Why's our monitor labelling this an incident or hazard?

The event involves AI systems (AI scribes) used in medical consultations that have produced incorrect outputs (wrong drug names, misdiagnoses) leading to direct harm to patients' health and safety. The AI system's malfunction or errors have directly caused or contributed to these harms, fulfilling the criteria for an AI Incident. The article details multiple cases where harm has occurred, not just potential risk, and highlights systemic issues with oversight and safety. Hence, the classification as AI Incident is appropriate.[AI generated]
AI principles
SafetyRobustness & digital security

Industries
Healthcare, drugs, and biotechnology

Affected stakeholders
Consumers

Harm types
Physical (injury)Psychological

Business function:
Other

AI system task:
Recognition/object detectionContent generation


Articles about this incident or hazard