Tesla urged to revise emergency braking AI after fatal 2021 Paris crash

Thumbnail Image

The information displayed in the AIM should not be reported as representing the official views of the OECD or of its member countries.

In December 2021, a Tesla Model 3 taxi’s Autonomous Emergency Braking System unexpectedly activated without obstacle, prompting the driver to press the accelerator. The vehicle slammed into a cyclist at 118 km/h, killing them and injuring about 20 people. France’s BEA-TT has recommended Tesla update its AI braking system and regulators amend UN rules.[AI generated]

Why's our monitor labelling this an incident or hazard?

The emergency braking system is an AI system as it autonomously detects obstacles and applies brakes. Its malfunction (false activation without obstacle) led the driver to react by pressing the accelerator, which contributed to the fatal accident. The harm includes death of a cyclist and injuries to others, fulfilling the criteria for an AI Incident. The report's recommendations to Tesla and regulators further confirm the AI system's role in the incident.[AI generated]
AI principles
SafetyRobustness & digital securityTransparency & explainabilityAccountabilityRespect of human rightsHuman wellbeing
Industries
Mobility and autonomous vehiclesGovernment, security, and defence
Affected stakeholders
General public
Harm types
Physical (death)Physical (injury)
Severity
AI incident
AI system task:
Recognition/object detectionEvent/anomaly detectionGoal-driven organisation

Articles about this incident or hazard

Thumbnail Image

Tesla appelé à modifier son système de freinage d'urgence après un accident mortel en 2021

2024-11-21
La Libre.be
Why's our monitor labelling this an incident or hazard?
The emergency braking system is an AI system as it autonomously detects obstacles and applies brakes. Its malfunction (false activation without obstacle) led the driver to react by pressing the accelerator, which contributed to the fatal accident. The harm includes death of a cyclist and injuries to others, fulfilling the criteria for an AI Incident. The report's recommendations to Tesla and regulators further confirm the AI system's role in the incident.
Thumbnail Image

Après un accident mortel causé par un taxi, Tesla est appelé à modifier son système de freinage d'urgence

2024-11-21
LaProvence.com
Why's our monitor labelling this an incident or hazard?
The event explicitly mentions the activation of Tesla's emergency braking system, an AI system designed to assist driving safety. The system's unexpected activation caused the driver to mistakenly press the accelerator, leading to a fatal crash with multiple injuries. Although the Autopilot was not active, the AEBS is an AI system whose malfunction directly contributed to the harm. Therefore, this qualifies as an AI Incident due to the AI system's malfunction indirectly causing injury and death.
Thumbnail Image

Pourquoi l'accident mortel survenu à Paris avec une Tesla en 2021 refait-il surface ?

2024-11-22
Auto Plus
Why's our monitor labelling this an incident or hazard?
The Tesla vehicle's AEBS is an AI system designed to detect obstacles and automatically brake to prevent collisions. The report states that the AEBS activated unexpectedly without an obstacle, which caused the driver to react incorrectly by pressing the accelerator, leading to a fatal accident. This shows a malfunction of an AI system that directly contributed to harm (death and injuries). The involvement of the AI system in the accident's causation and the resulting harm qualifies this event as an AI Incident. The legal and regulatory responses further confirm the significance of the AI system's role in the harm.
Thumbnail Image

Accident mortel à Paris en 2021 : Tesla appelé à modifier son système de freinage d'urgence | TF1 INFO

2024-11-22
TF1 INFO
Why's our monitor labelling this an incident or hazard?
The emergency braking system is an AI system designed to detect obstacles and automatically brake. Its unexpected activation without an obstacle caused the driver to mistakenly press the accelerator, leading to a fatal accident. This is a direct harm caused by the malfunction of an AI system. Therefore, this qualifies as an AI Incident due to injury and death caused by the AI system's malfunction and design flaws.