Epic Sepsis Prediction Algorithm Fails to Accurately Detect Deadly Infections in Hospitals

Thumbnail Image

The information displayed in the AIM should not be reported as representing the official views of the OECD or of its member countries.

A widely used AI algorithm by Epic Systems, designed to predict sepsis in hospital patients, was found to miss most cases and generate frequent false alarms, according to a University of Michigan study. The tool’s poor accuracy risks delayed treatment, unnecessary interventions, and contributes to clinician alert fatigue, potentially harming patient care.[AI generated]

Why's our monitor labelling this an incident or hazard?

The Epic sepsis prediction algorithm is an AI system used in hospitals to identify a deadly condition early. The study shows it misses most sepsis cases and has a high false positive rate, which can lead to harm by delaying treatment or causing unnecessary interventions. The AI system's poor performance and flawed development approach directly contribute to harm to patients' health, fulfilling the criteria for an AI Incident. The article documents realized harm rather than just potential risk, and the AI system's malfunction is central to the event.[AI generated]
AI principles
SafetyRobustness & digital securityAccountability

Industries
Healthcare, drugs, and biotechnology

Affected stakeholders
ConsumersWorkers

Harm types
Physical (injury)Physical (death)Psychological

Severity
AI incident

Business function:
Monitoring and quality control

AI system task:
Forecasting/prediction

In other databases

Articles about this incident or hazard

Thumbnail Image

A hospital algorithm designed to predict a deadly condition misses most cases

2021-06-22
The Verge
Why's our monitor labelling this an incident or hazard?
The Epic sepsis prediction algorithm is an AI system used in hospitals to identify a deadly condition early. The study shows it misses most sepsis cases and has a high false positive rate, which can lead to harm by delaying treatment or causing unnecessary interventions. The AI system's poor performance and flawed development approach directly contribute to harm to patients' health, fulfilling the criteria for an AI Incident. The article documents realized harm rather than just potential risk, and the AI system's malfunction is central to the event.
Thumbnail Image

An Algorithm That Predicts Deadly Infections Is Often Flawed

2021-06-21
Wired
Why's our monitor labelling this an incident or hazard?
The event involves an AI system explicitly described as a proprietary algorithm used for early detection of sepsis in hospital patients. The study shows the AI system's malfunction—missing many sepsis cases and producing false alarms—directly impacts patient health outcomes, which is harm to persons. The AI system's poor performance could lead to injury or harm to patients due to missed or delayed treatment. Therefore, this qualifies as an AI Incident under the definition of harm caused by AI system malfunction in healthcare.
Thumbnail Image

Study Questions Accuracy of Widely Used Sepsis Prediction Tool

2021-06-24
Medscape
Why's our monitor labelling this an incident or hazard?
The Epic Sepsis Model is an AI system designed to predict sepsis onset. The study shows that its use has directly led to harm by missing many patients who developed sepsis and generating many false alerts, which can cause alert fatigue and potentially delay appropriate treatment. This constitutes injury or harm to the health of patients, fulfilling the criteria for an AI Incident. The article focuses on the AI system's performance and its implications for patient health, not just general information or complementary updates.
Thumbnail Image

Clinical Benefit of Epic Sepsis Model in Question - Drugs.com MedNews

2021-06-22
Drugs.com
Why's our monitor labelling this an incident or hazard?
The event involves the use of an AI system (the Epic Sepsis Model) in clinical decision-making. The study demonstrates that the AI system's poor sensitivity and calibration have directly led to potential harm by failing to identify many sepsis cases and causing alert fatigue, which can disrupt clinical management and delay treatment. This constitutes an AI Incident because the AI system's use has directly led to harm to patient health through missed or delayed treatment and operational disruption in clinical care.
Thumbnail Image

External Validation of a Widely Implemented Sepsis Prediction Model in Hospitalized Patients

2021-06-23
jamanetwork.com
Why's our monitor labelling this an incident or hazard?
The ESM is an AI system designed to predict sepsis onset. The study shows that its use has directly led to poor identification of sepsis cases (only 7% sensitivity for patients not receiving timely antibiotics) and a high number of false alerts (18% of hospitalizations), which can cause alert fatigue among clinicians. These factors indicate that the AI system's malfunction or poor performance has directly led to harm to patients' health by failing to timely identify sepsis, a life-threatening condition. Therefore, this event qualifies as an AI Incident due to injury or harm to health caused by the AI system's use and malfunction.
Thumbnail Image

Popular sepsis prediction tool less accurate than claimed

2021-06-21
Medical Xpress - Medical and Health News
Why's our monitor labelling this an incident or hazard?
The sepsis prediction tool is an AI system used to predict patient risk for sepsis. The article details how the model's development and use have led to poor predictive performance, resulting in many false positives and missed cases. This directly impacts patient care quality and safety, as timely recognition and treatment of sepsis are critical. The harm is indirect but material, as clinicians relying on the tool may be misled, leading to potential injury or harm to patients. Therefore, this event qualifies as an AI Incident due to realized harm to health caused by the AI system's use and malfunction.
Thumbnail Image

Epic's widely used sepsis prediction model performs worse than claimed, research finds

2021-06-21
Hospital Review
Why's our monitor labelling this an incident or hazard?
The sepsis prediction model is an AI system used in clinical settings to predict patient risk. The study shows the model's performance is significantly lower than claimed, which could lead to missed or delayed sepsis diagnoses, directly impacting patient health (harm to persons). This constitutes an AI Incident because the AI system's use has directly led to harm or risk of harm through inaccurate predictions. The article reports realized performance issues and their implications, not just potential future harm, so it is an AI Incident rather than a hazard or complementary information.
Thumbnail Image

Popular sepsis prediction tool less accurate than claimed

2021-06-21
EurekAlert!
Why's our monitor labelling this an incident or hazard?
The sepsis prediction tool is an AI system used in clinical settings to predict patient risk. The study shows that the model's inaccurate predictions (false positives and timing issues) could lead to suboptimal clinical decisions, delayed or missed treatment, and unnecessary interventions, all of which constitute harm to patient health. The harm is indirect but significant, as the AI system's flawed outputs influence clinical actions. The article also highlights the need for regulatory oversight and validation, reinforcing the seriousness of the issue. Hence, this is an AI Incident rather than a hazard or complementary information.
Thumbnail Image

Algorithm used to predict sepsis infections in hundreds of US hospitals isn't as good as maker claims - study

2021-06-23
theregister.com
Why's our monitor labelling this an incident or hazard?
The algorithm is an AI system used for clinical prediction. Its development and use have directly led to harm by providing inaccurate sepsis risk predictions, which can cause mismanagement of patient care and increased risk of death. The study documents realized harm (inaccuracy leading to poor clinical outcomes) rather than just potential harm. Therefore, this qualifies as an AI Incident due to harm to health of patients resulting from the AI system's use and malfunction (inaccuracy).
Thumbnail Image

Popular sepsis prediction model works 'substantially worse' than claimed, researchers find

2021-06-23
MedCity News
Why's our monitor labelling this an incident or hazard?
The Epic Sepsis Model is an AI system that predicts sepsis risk using patient data. The study found that the model's accuracy is significantly lower than claimed, causing many false alerts that contribute to alert fatigue among clinicians. Alert fatigue can impair clinical decision-making and patient care, constituting indirect harm to health. The AI system's malfunction (poor predictive performance) is directly linked to this harm. Hence, the event meets the criteria for an AI Incident due to realized harm caused by the AI system's use and malfunction.
Thumbnail Image

A Hospital Algorithm Designed To Predict a Deadly Condition Misses Most Cases (Slashdot)

2021-06-23
Tech Investor News
Why's our monitor labelling this an incident or hazard?
The Epic sepsis prediction tool is an AI system used in clinical settings to identify patients at risk of sepsis. The study shows that the algorithm's poor performance leads to missed sepsis cases and false alerts, which can directly harm patients by delaying treatment or causing unnecessary interventions. This constitutes harm to health (a), and the AI system's malfunction or inadequate development is a contributing factor. Therefore, this qualifies as an AI Incident.
Thumbnail Image

Study Shows Algorithm That Predicts Deadly Hospital Infections is Far From Perfect -- And Other A.I. Health Woes

2021-06-24
USSA News
Why's our monitor labelling this an incident or hazard?
The AI system in question is an algorithm used in hospitals to predict sepsis, a deadly infection. The study shows it missed two-thirds of cases and frequently issued false alarms, indicating malfunction or poor performance. This directly relates to harm to health (injury or death) of patients, fulfilling the criteria for an AI Incident. The article also references previous similar harms caused by AI health algorithms, reinforcing the classification. The AI system's use and malfunction have directly led to harm or risk of harm, making this an AI Incident rather than a hazard or complementary information.
Thumbnail Image

Un algoritmo para prevenir la sepsis revela que queda mucho por hacer en la salud digital

2021-06-23
Business Insider
Why's our monitor labelling this an incident or hazard?
The AI system is explicitly mentioned as an algorithm designed to identify sepsis signs. The study shows it only correctly identified sepsis 63% of the time and missed many cases, indicating malfunction or poor performance. Since sepsis detection is vital to prevent mortality, the AI's failure to detect cases represents a direct or indirect contribution to harm to patients' health. This fits the definition of an AI Incident because the AI system's malfunction has led to harm (or at least significant risk of harm) in a healthcare context.
Thumbnail Image

Un algoritmo hospitalario diseñado para predecir una enfermedad mortal pasa por alto la mayoría de los casos - Es de Latino News

2021-06-22
Es de Latino, Noticias en español para Latinos.
Why's our monitor labelling this an incident or hazard?
The event involves an AI system explicitly described as a predictive algorithm for sepsis detection used in hospitals. The system's malfunction—missing most sepsis cases and generating many false positives—directly leads to harm to patients' health by potentially delaying treatment for a life-threatening condition. The article provides evidence of realized harm (missed sepsis cases) rather than just potential risk. Hence, it meets the criteria for an AI Incident under the definition of injury or harm to health caused by the AI system's use and malfunction.
Thumbnail Image

Un algoritmo diseñado para predecir una enfermedad mortal falló en cuatro de cada diez casos

2021-06-23
Todo Noticias
Why's our monitor labelling this an incident or hazard?
The event involves an AI system explicitly described as an automated predictive tool for sepsis detection in hospitals. The system's malfunction—its failure to accurately predict sepsis cases—has directly led to harm to patients' health, fulfilling the criteria for an AI Incident. The article provides evidence of realized harm (missed diagnoses of a deadly condition), not just potential harm, and discusses the need for transparency and monitoring, reinforcing the significance of the incident. Hence, the classification as AI Incident is appropriate.
Thumbnail Image

Un algoritmo para detectar pacientes con sepsis genera una diatriba en Estados Unidos

2021-06-24
FayerWayer
Why's our monitor labelling this an incident or hazard?
The Epic Systems sepsis prediction model is an AI system used to infer patient risk and influence clinical decisions. The article details a study showing the model's poor accuracy, including many missed sepsis cases and false alarms, which can lead to delayed treatment or unnecessary interventions. This directly impacts patient health, fulfilling the criterion of harm to persons. The AI system's use and its flawed outputs have directly contributed to these risks, constituting an AI Incident. The article does not merely warn of potential harm but documents actual performance issues and clinical concerns, so it is not merely an AI Hazard or Complementary Information. Hence, the classification is AI Incident.