AITO M5 Intelligent Driving Demo Causes Crash and Injuries During Test Drive

Thumbnail Image

The information displayed in the AIM should not be reported as representing the official views of the OECD or of its member countries.

During a test drive in Hubei, China, an AITO M5 vehicle's intelligent driving (autonomous) system failed to brake when a bus turned ahead, resulting in a collision. The sales representative had both hands off the wheel, relying on the AI system. The crash caused significant injuries to the customer and passenger.[AI generated]

Why's our monitor labelling this an incident or hazard?

The vehicle was in intelligent driving mode, which implies the use of an AI system for autonomous or assisted driving. The accident occurred because the AI system did not take timely emergency braking action, leading to a collision and physical injuries. This is a direct harm caused by the AI system's malfunction or failure to act appropriately. Therefore, this qualifies as an AI Incident under the definition of harm to a person caused directly or indirectly by the use or malfunction of an AI system.[AI generated]
AI principles
SafetyRobustness & digital securityAccountabilityTransparency & explainabilityHuman wellbeing

Industries
Mobility and autonomous vehicles

Affected stakeholders
Consumers

Harm types
Physical (injury)

Business function:
Sales

AI system task:
Recognition/object detectionForecasting/predictionEvent/anomaly detectionGoal-driven organisation


Articles about this incident or hazard