Zeekr Executive Warns Against Unsafe Use of Autonomous Driving After Viral Video

Thumbnail Image

The information displayed in the AIM should not be reported as representing the official views of the OECD or of its member countries.

A viral video showed Zeekr car owners lying down and watching TV while the vehicle operated in autonomous mode, bypassing hand detection with a bottle. Zeekr's CMO, Guan Haitao, publicly discouraged such unsafe and potentially illegal behavior, emphasizing the need for users to follow safety regulations when using AI driving features.[AI generated]

Why's our monitor labelling this an incident or hazard?

An AI system (automated driving) is involved, and the misuse of the system (driver lying down and not monitoring) could plausibly lead to harm such as accidents or injury. However, no actual harm is reported in the event, only a warning against the behavior. Therefore, this qualifies as an AI Hazard due to the plausible risk of harm from misuse of the AI system.[AI generated]
AI principles
SafetyRobustness & digital securityAccountabilityDemocracy & human autonomy

Industries
Mobility and autonomous vehiclesConsumer products

Affected stakeholders
ConsumersGeneral public

Harm types
Physical (injury)Physical (death)ReputationalEconomic/Property

Severity
AI hazard

Business function:
Monitoring and quality control

AI system task:
Recognition/object detectionGoal-driven organisation


Articles about this incident or hazard

Thumbnail Image

一车主开启自动驾驶后竟直接躺下 极氪关海涛:非常不鼓励类似违法行为 - cnBeta.COM 移动版

2024-10-07
cnBeta.COM
Why's our monitor labelling this an incident or hazard?
An AI system (automated driving) is involved, and the misuse of the system (driver lying down and not monitoring) could plausibly lead to harm such as accidents or injury. However, no actual harm is reported in the event, only a warning against the behavior. Therefore, this qualifies as an AI Hazard due to the plausible risk of harm from misuse of the AI system.
Thumbnail Image

极氪高管回应车主在汽车自动驾驶时盖被躺平 不鼓励类似行为

2024-10-08
k.sina.com.cn
Why's our monitor labelling this an incident or hazard?
An AI system (the autonomous driving system) is explicitly involved, and its use in this scenario is directly linked to a risky behavior that could plausibly lead to harm (e.g., injury or death) if the system fails or encounters a situation it cannot handle. Although no harm has yet occurred, the behavior of lying down and not monitoring the system while it is engaged creates a credible risk of an AI Incident. Therefore, this event qualifies as an AI Hazard because it plausibly could lead to injury or harm due to misuse of the AI system's capabilities.
Thumbnail Image

一车主开启自动驾驶后竟直接躺下!极氪关海涛:非常不鼓励类似违法行为

2024-10-07
驱动之家
Why's our monitor labelling this an incident or hazard?
An AI system (automatic driving system) is involved, and the event concerns the use of this AI system. However, no actual harm or incident has been reported; rather, the event highlights a risky behavior that could plausibly lead to harm if continued. Since no harm has occurred yet but there is a credible risk of harm due to misuse of the AI system, this qualifies as an AI Hazard.
Thumbnail Image

极氪高管回应车主在汽车自动驾驶时盖被躺平 不鼓励类似行为

2024-10-07
China Finance Online
Why's our monitor labelling this an incident or hazard?
The article involves an AI system (autonomous driving) being used in a way that could plausibly lead to harm, as the occupants are not monitoring the vehicle, which is unsafe and could cause accidents. The company's response discouraging the behavior further supports the recognition of potential risk. Since no actual harm or incident has occurred yet, this qualifies as an AI Hazard rather than an AI Incident.
Thumbnail Image

极氪关海涛回应车主在汽车自动驾驶时盖被躺下看电视:非常不鼓励类似行为

2024-10-06
新浪财经
Why's our monitor labelling this an incident or hazard?
An AI system (the car's autonomous driving feature) is explicitly involved. The event concerns the use of this AI system and the potential for harm due to unsafe user behavior (lying down and not paying attention while the car is driving). Although no harm has occurred yet, the behavior could plausibly lead to injury or harm if the system fails or requires human intervention. The company's response discouraging this behavior is a complementary update but the main event is the plausible risk of harm from misuse of the AI system, qualifying this as an AI Hazard.
Thumbnail Image

车主在汽车自动驾驶时盖被躺下,极氪关海涛:非常不鼓励类似行为

2024-10-06
k.sina.com.cn
Why's our monitor labelling this an incident or hazard?
An AI system (the car's automatic driving system) is involved, and the misuse of the system by the user (bypassing hand detection and not paying attention) could plausibly lead to harm such as accidents or injury. However, no actual harm or incident is reported in the article, only a warning against the behavior. Therefore, this qualifies as an AI Hazard because the misuse could plausibly lead to an AI Incident but has not yet caused harm.
Thumbnail Image

车主自动驾驶时盖被躺下看电视,极氪关海涛回应

2024-10-06
k.sina.com.cn
Why's our monitor labelling this an incident or hazard?
An AI system (automated driving) is involved, and the misuse of the system by the user (lying down and not paying attention) creates a direct risk of harm to the driver and others. Although no harm has yet occurred, the behavior plausibly could lead to injury or accidents due to overreliance or misuse of the AI system. Therefore, this event qualifies as an AI Hazard because it plausibly could lead to an AI Incident (harm).
Thumbnail Image

一车主开启自动驾驶后竟直接躺下!极氪关海涛:非常不鼓励类似违法行为

2024-10-07
新浪财经
Why's our monitor labelling this an incident or hazard?
An AI system (automatic driving system) is involved, and the misuse (lying down and not paying attention) could plausibly lead to harm (accidents or injury). However, no actual harm or incident is reported. The company's response is a warning and call for safe use, indicating potential future risk rather than a realized incident. Therefore, this qualifies as an AI Hazard.