The information displayed in the AIM should not be reported as representing the official views of the OECD or of its member countries.
During December test flights in Oregon, two Amazon Prime Air drones crashed due to an AI malfunction. Faulty LiDAR sensor readings misinterpreted rain as ground contact, triggering a mid-air shutdown of the propellers. The removal of backup sensors contributed to the error, leading to the destruction of the drones.[AI generated]
Why's our monitor labelling this an incident or hazard?
The drones are autonomous AI systems relying on sensor inputs and software to make real-time decisions. The software update increased sensor sensitivity, and the removal of a physical fail-safe sensor led to incorrect altitude readings, causing the drones to shut down mid-flight and crash. This malfunction directly caused harm to property (the drones themselves) and disrupted the operation of the delivery system, which is critical infrastructure for Amazon's logistics. The event involves the use and malfunction of AI systems leading to realized harm, fitting the definition of an AI Incident rather than a hazard or complementary information.[AI generated]
