Waymo Autonomous Vehicle Fails to Stop During Hazardous Teen Behavior

Thumbnail Image

The information displayed in the AIM should not be reported as representing the official views of the OECD or of its member countries.

In Santa Monica, a Waymo autonomous vehicle continued driving while several teens hung out of its windows, risking injury. Despite witnesses alerting Waymo, the AI system did not stop the car, highlighting a failure in safety protocols and raising concerns about the system's ability to prevent harm.[AI generated]

Why's our monitor labelling this an incident or hazard?

The event explicitly involves an AI system (Waymo's autonomous vehicle) whose use was misappropriated by children, creating a hazardous situation on public roads. The AI system's operation directly contributed to a risk of harm to the children and others. The company's failure to stop the vehicle despite being alerted exacerbates the risk. Although no actual injury is reported, the incident involves direct risk of injury or harm to persons, which meets the criteria for an AI Incident. The AI system's malfunction or misuse led to a safety hazard with potential for injury, fulfilling the definition of an AI Incident rather than a mere hazard or complementary information.[AI generated]
AI principles
SafetyRobustness & digital security

Industries
Mobility and autonomous vehicles

Affected stakeholders
Consumers

Harm types
Physical (injury)

Business function:
Logistics

AI system task:
Recognition/object detectionGoal-driven organisation


Articles about this incident or hazard