
The information displayed in the AIM should not be reported as representing the official views of the OECD or of its member countries.
Waymo's autonomous vehicle AI left passengers trapped and vulnerable during attacks by anti-AI individuals in San Francisco. The AI's cautious programming prevented the vehicle from escaping, exposing passengers to harm. The lack of remote override or human control exacerbated the safety risk.[AI generated]
Why's our monitor labelling this an incident or hazard?
The event involves an AI system (Waymo's autonomous driving system) whose use directly led to a dangerous traffic situation that endangered the health and safety of passengers and other drivers. The AI system's attempt to make an unprotected left turn across oncoming traffic on a complex road without traffic signals is a known challenge for autonomous vehicles and here resulted in a near-accident scenario. The passengers' fear and the forced braking of other vehicles demonstrate direct harm or risk of harm. Therefore, this qualifies as an AI Incident due to the direct involvement of an AI system causing or nearly causing injury or harm to people.[AI generated]