Apple's Crash Detection AI Causes Emergency Service Disruptions Due to False Alarms

Thumbnail Image

The information displayed in the AIM should not be reported as representing the official views of the OECD or of its member countries.

Apple's AI-powered Crash Detection feature on iPhone 14 and Apple Watch has triggered numerous false emergency calls during activities like skiing and amusement park rides, straining emergency services. Apple updated its guidance, urging users not to hang up on unintended calls but to inform responders if help is not needed.[AI generated]

Why's our monitor labelling this an incident or hazard?

The event involves an AI system (Crash Detection feature using accelerometer and gyroscope sensors with algorithms to detect crashes) whose use has indirectly led to harm by causing numerous false emergency calls, wasting emergency responders' time and resources, which is a disruption of critical infrastructure management (emergency services). Therefore, this qualifies as an AI Incident. The article also describes Apple's mitigation efforts, but the primary focus is on the harm caused by the AI system's use and its impact on emergency services.[AI generated]
AI principles
Robustness & digital securitySafetyAccountabilityTransparency & explainabilityDemocracy & human autonomy

Industries
Consumer productsGovernment, security, and defenceHealthcare, drugs, and biotechnology

Affected stakeholders
WorkersGovernmentGeneral public

Harm types
Public interestEconomic/PropertyReputational

Severity
AI incident

Business function:
Other

AI system task:
Event/anomaly detection


Articles about this incident or hazard

Thumbnail Image

Apple: Apple issues user guidelines for Accidental Crash Detection Calls - Times of India

2023-04-07
The Times of India
Why's our monitor labelling this an incident or hazard?
The event involves an AI system (Crash Detection feature using accelerometer and gyroscope sensors with algorithms to detect crashes) whose use has indirectly led to harm by causing numerous false emergency calls, wasting emergency responders' time and resources, which is a disruption of critical infrastructure management (emergency services). Therefore, this qualifies as an AI Incident. The article also describes Apple's mitigation efforts, but the primary focus is on the harm caused by the AI system's use and its impact on emergency services.
Thumbnail Image

Apple advises what to do if iPhone warning is triggered on your phone

2023-04-07
LADbible
Why's our monitor labelling this an incident or hazard?
The 'Car Crash Detection' feature involves AI systems that infer from sensor data to detect accidents and automatically initiate emergency calls. The false positive activations have directly led to disruption of emergency service operations, which qualifies as harm under the disruption of critical infrastructure category. Since the AI system's malfunction (false triggering) has caused real-world negative impacts, this event meets the criteria for an AI Incident.
Thumbnail Image

Apple Tells Users Not to Hang up on Operators in Accidental 911 Calls

2023-04-07
www.theepochtimes.com
Why's our monitor labelling this an incident or hazard?
The Crash Detection feature involves AI systems processing sensor data to detect crashes and initiate emergency calls. However, the article only discusses the feature's operation and Apple's advisory to users on how to respond to accidental calls. There is no report of injury, disruption, rights violation, or other harm caused by the AI system, nor any credible risk of such harm described. Therefore, this is not an AI Incident or AI Hazard. The article provides complementary information about the AI system's functioning and user guidance, fitting the definition of Complementary Information.
Thumbnail Image

If Your iPhone Calls for Help, Don't Hang Up

2023-04-06
Gizmodo
Why's our monitor labelling this an incident or hazard?
The crash detection feature is an AI system that infers from sensor data to decide when to call emergency services. Its malfunction (false positives) has directly led to harm by causing a significant number of unnecessary emergency calls, disrupting emergency response operations, which is a form of harm to critical infrastructure management. The article details realized harm (false alarms causing operational issues) and Apple's responses, confirming this is an AI Incident rather than a mere hazard or complementary information.
Thumbnail Image

Apple reveals the one thing you shouldn't do if your iPhone accidentally calls 911

2023-04-06
iMore
Why's our monitor labelling this an incident or hazard?
The event involves an AI system (Crash Detection) that automatically calls emergency services based on inferred crash detection. The system's use has led to false emergency calls, which can cause disruption to emergency services (harm category b: disruption of critical infrastructure management). Although no direct physical harm is described, the false calls impose a burden on emergency response systems, which is a recognized harm. Therefore, this qualifies as an AI Incident due to the AI system's use directly leading to harm (disruption).
Thumbnail Image

Don't hang up on emergency services if your iPhone automatically calls them

2023-04-06
Phone Arena
Why's our monitor labelling this an incident or hazard?
The Crash Detection feature involves AI systems that detect crashes and automatically call emergency services. The false positives have directly led to disruption of emergency services, which qualifies as harm under the disruption of critical infrastructure category. Therefore, this event is an AI Incident due to the AI system's malfunction causing real-world harm.
Thumbnail Image

If your iPhone calls emergency services, don't hang up if you're fine says Apple | AppleInsider

2023-04-06
AppleInsider
Why's our monitor labelling this an incident or hazard?
The iPhone 14 Crash Detection feature is an AI system that analyzes sensor data to detect crashes and initiate emergency calls. The false alarms and users missing the alert cause unnecessary emergency calls, which strain emergency services, thus disrupting critical infrastructure management. Although no physical injury is reported, the disruption to emergency services qualifies as harm under the framework. Therefore, this event is an AI Incident due to the AI system's malfunction indirectly causing harm.
Thumbnail Image

iPhone 14 Crash Detection: Apple Urges Users to NOT Cancel Unintended Emergency Call

2023-04-06
Tech Times
Why's our monitor labelling this an incident or hazard?
The iPhone 14 Crash Detection feature is an AI system that automatically detects crashes and calls emergency services. The article reports on false activations causing unintended emergency calls, which is a malfunction of the AI system. However, no direct harm such as injury, violation of rights, or critical disruption is reported. The article mainly provides updated guidance from Apple to users on how to handle false calls and highlights the feature's life-saving potential. This fits the definition of Complementary Information, as it updates understanding and responses to an existing AI system's behavior without describing a new incident or hazard causing harm or plausible future harm.
Thumbnail Image

Apple updates Crash Detection support document regarding dropped calls

2023-04-08
iLounge
Why's our monitor labelling this an incident or hazard?
Crash Detection is an AI-enabled feature that detects collisions and automatically calls emergency services, thus involving an AI system. However, the update is about guidance to users on how to handle calls to reduce false alarms and does not describe any actual harm or incident caused by the AI system. There is no indication of injury, rights violation, or other harm resulting from the AI's use or malfunction. The update aims to improve the system's effectiveness and reduce unnecessary emergency responses, which is a complementary improvement rather than an incident or hazard.
Thumbnail Image

Apple to users on Crash Detection: Don't hang up if accidental call is made | WRAL TechWire

2023-04-06
WRAL TechWire
Why's our monitor labelling this an incident or hazard?
The Crash Detection feature involves AI to detect collisions and trigger emergency calls. The accidental activations causing false emergency calls represent a malfunction or unintended use of the AI system, which can lead to disruption of emergency services and potential harm if resources are misallocated. However, the article does not report actual harm occurring from these false calls, only complaints and Apple's mitigation efforts. Therefore, this event represents an AI Hazard, as the AI system's malfunction could plausibly lead to harm (e.g., emergency service disruption), but no direct harm is reported yet.