NHTSA Investigates Comma.ai Driver Assistance Systems After Fatal Crashes

Thumbnail Image

The information displayed in the AIM should not be reported as representing the official views of the OECD or of its member countries.

The U.S. National Highway Traffic Safety Administration is investigating five crashes involving comma.ai's AI-based driver-assistance systems, which failed to detect stopped or slow vehicles, resulting in three deaths and multiple injuries. Some incidents involved modified versions of the open-source software, raising liability questions.[AI generated]

Why's our monitor labelling this an incident or hazard?

The article explicitly mentions that the AI system (Comma AI's driver-assistance software) was engaged during multiple crashes resulting in three deaths and eleven injuries. The AI system's failure to detect or respond to vehicles in the lane is a direct cause of harm. This fits the definition of an AI Incident, as the AI system's malfunction and use have directly led to injury and death, fulfilling harm criterion (a).[AI generated]
AI principles
SafetyRobustness & digital security

Industries
Mobility and autonomous vehicles

Affected stakeholders
Consumers

Harm types
Physical (death)Physical (injury)

AI system task:
Recognition/object detection


Articles about this incident or hazard