AI Driving Assistance Failure Leads to Rear-End Collision During Test Drive

Thumbnail Image

The information displayed in the AIM should not be reported as representing the official views of the OECD or of its member countries.

A test drive of BYD's Denza vehicle, equipped with an AI-based driving assistance system, resulted in a rear-end collision. The sales representative instructed the driver to rely on the system, which failed to stop the car in time. The incident highlights the risks of over-reliance on AI systems in vehicles.[AI generated]

Why's our monitor labelling this an incident or hazard?

During a Densee (腾势) test drive, the salesperson instructed the user to remove hands from the steering wheel, relying on the vehicle’s AI-based intelligent driving system to handle braking. The system failed to recognize the vehicle ahead and did not stop, resulting in a collision. This is a direct harm (property damage and safety risk) caused by an AI system malfunction in use.[AI generated]
AI principles
SafetyRobustness & digital securityAccountabilityTransparency & explainabilityDemocracy & human autonomy

Industries
Mobility and autonomous vehicles

Affected stakeholders
Consumers

Harm types
Physical (injury)Economic/PropertyReputational

Severity
AI incident

Business function:
SalesResearch and developmentMonitoring and quality control

AI system task:
Recognition/object detectionGoal-driven organisation


Articles about this incident or hazard

Thumbnail Image

网传腾势销售要求试驾用户双手离开方向盘,智驾失误导致车辆追尾

2024-07-02
finance.sina.com.cn
Why's our monitor labelling this an incident or hazard?
During a Densee (腾势) test drive, the salesperson instructed the user to remove hands from the steering wheel, relying on the vehicle’s AI-based intelligent driving system to handle braking. The system failed to recognize the vehicle ahead and did not stop, resulting in a collision. This is a direct harm (property damage and safety risk) caused by an AI system malfunction in use.
Thumbnail Image

试驾腾势测试车辆AEB功能发生追尾?知情人士称系司机误触致智能领航功能退出

2024-07-02
凤凰网(凤凰新媒体)
Why's our monitor labelling this an incident or hazard?
The incident involves an AI-based driving assistance system (AEB/intelligent navigation). Its deactivation or malfunction directly contributed to a property-damaging collision. An AI system’s failure to brake as expected resulted in real-world harm (vehicle damage), meeting the definition of an AI Incident.
Thumbnail Image

【图】腾势试驾车开启驾驶辅助后发生追尾事故_汽车之家

2024-07-02
汽车之家(Autohome.com.cn)
Why's our monitor labelling this an incident or hazard?
The collision occurred due to the misuse and misunderstanding of the L2 driver assistance system (an AI system) and its disengagement logic. The AI system’s deployment and the sales rep’s instructions directly contributed to the crash, causing property damage. This meets the criteria for an AI Incident, as the AI system’s behavior and user interaction led to harm.
Thumbnail Image

网传腾势销售要求试驾用户双手离开方向盘,智驾失误导致车辆追尾-科技频道-和讯网

2024-07-02
和讯网
Why's our monitor labelling this an incident or hazard?
The incident involves a malfunction of an AI system (the ‘intelligent driving’ feature) during its use; the system’s failure to recognize an obstacle and brake directly caused property damage (a collision). Thus it constitutes an AI Incident.
Thumbnail Image

试驾腾势测试车辆AEB功能发生追尾?知情人士称系司机误触致智能领航功能退出

2024-07-02
app.myzaker.com
Why's our monitor labelling this an incident or hazard?
The event involves an AI-driven safety system (AEB and intelligent pilot) whose deactivation—triggered by the driver—suppressed emergency braking and directly led to a rear-end collision. The AI system’s malfunction or inhibited function caused the harm, fitting the definition of an AI Incident.
Thumbnail Image

男子试驾腾势汽车,测试AEB功能却发生追尾?多方回应 2024-07-03 09:07

2024-07-03
sznews.com
Why's our monitor labelling this an incident or hazard?
The AEB system is an AI system designed to prevent collisions by automatically braking when obstacles are detected. The accident happened during testing of this AI system, and the failure to stop led directly to a collision, causing harm. The involvement of the AI system's malfunction or suppression (due to driver actions) is central to the incident. The event describes realized harm (a collision and operational disruption), so it meets the criteria for an AI Incident rather than a hazard or complementary information.
Thumbnail Image

腾势汽车销售让客户放开方向盘致追尾_手机网易网

2024-07-02
m.163.com
Why's our monitor labelling this an incident or hazard?
The vehicle's automatic emergency braking is an AI system designed to assist driving and prevent collisions. The incident directly resulted from the AI system's failure to act appropriately in a critical situation, compounded by the sales representative's encouragement to disengage manual control. This caused a collision, fulfilling the criteria for an AI Incident due to direct harm caused by the AI system's malfunction and misuse during its use.
Thumbnail Image

网传腾势销售要求试驾用户双手离开方向盘 智驾失误导致车辆追尾 - cnBeta.COM 移动版

2024-07-02
cnBeta.COM
Why's our monitor labelling this an incident or hazard?
The incident involves an AI system (intelligent driving/autonomous driving) whose malfunction during use directly led to a vehicle collision (harm to property and potential harm to persons). The AI system's failure to properly control the vehicle and prevent the crash constitutes an AI Incident as per the definitions. The event is not merely a potential hazard or complementary information but a realized harm caused by AI malfunction.