Seattle Fire Department Secretly Uses AI to Triage 911 Calls, Leading to Delayed Emergency Responses

Thumbnail Image

The information displayed in the AIM should not be reported as representing the official views of the OECD or of its member countries.

The Seattle Fire Department used Corti AI to analyze all 911 medical calls since December 2023, routing some callers to a nurse line instead of dispatching ambulances. This was done without public disclosure or oversight, raising privacy concerns and contributing to delayed emergency responses and at least one wrongful death lawsuit.[AI generated]

Why's our monitor labelling this an incident or hazard?

An AI system is explicitly involved, analyzing live 911 medical calls and influencing dispatch decisions. The AI's use has directly or indirectly led to harm, including delayed ambulance responses and at least one wrongful death lawsuit, indicating harm to health and safety. The lack of disclosure and governance further exacerbates the risk and actual harm. These factors meet the criteria for an AI Incident, as the AI system's use has contributed to realized harm to people.[AI generated]
AI principles
Transparency & explainabilityPrivacy & data governance

Industries
Government, security, and defenceHealthcare, drugs, and biotechnology

Affected stakeholders
General public

Harm types
Physical (death)

Business function:
Citizen/customer service

AI system task:
Goal-driven organisation


Articles about this incident or hazard