Recall of 4,000 Xiaomi Drones Due to AI Flight Control Software Risk

Thumbnail Image

The information displayed in the AIM should not be reported as representing the official views of the OECD or of its member countries.

Beijing FIMI Technology is recalling 4,000 Xiaomi Mi 4K drones manufactured in February 2017 due to a flight control software malfunction. Improper parameter settings in the AI-driven system could cause drones to lose power and crash at high altitudes, posing injury risks. A software update is being provided to address the hazard.[AI generated]

Why's our monitor labelling this an incident or hazard?

The drone's flight control system software is an AI system or at least an AI-enabled system managing autonomous flight parameters. The malfunction (incorrect software control parameters) could directly lead to physical harm (injury from drone crashes). Since the issue has been identified and a recall and software update are underway, the event involves a malfunction leading to a safety hazard that has been realized as a risk. Because the harm is plausible and the malfunction is confirmed, this qualifies as an AI Incident due to the direct link between the AI system malfunction and potential physical harm.[AI generated]
AI principles
SafetyRobustness & digital security

Industries
Robots, sensors, and IT hardware

Affected stakeholders
ConsumersGeneral public

Harm types
Physical (injury)

Severity
AI incident

AI system task:
Goal-driven organisation


Articles about this incident or hazard

Thumbnail Image

小米无人机4K版部分被召回 存在高海拔飞行失去动力隐患

2021-07-18
chinaz.com
Why's our monitor labelling this an incident or hazard?
The drone's flight control system software is an AI system or at least an AI-enabled system managing autonomous flight parameters. The malfunction (incorrect software control parameters) could directly lead to physical harm (injury from drone crashes). Since the issue has been identified and a recall and software update are underway, the event involves a malfunction leading to a safety hazard that has been realized as a risk. Because the harm is plausible and the malfunction is confirmed, this qualifies as an AI Incident due to the direct link between the AI system malfunction and potential physical harm.
Thumbnail Image

飞米科技召回部分无人机 4K 版套装 高海拔极限飞行时可能失去动力 - cnBeta.COM 移动版

2021-07-16
cnBeta.COM
Why's our monitor labelling this an incident or hazard?
The drones' flight control system software is an AI system or at least an AI-enabled control system managing autonomous or semi-autonomous flight. The software parameter mismatch leading to loss of power and potential crashes constitutes a malfunction of the AI system that directly leads to harm (physical injury risk from drone crashes). Therefore, this event qualifies as an AI Incident due to the realized safety hazard and harm potential caused by the AI system malfunction.
Thumbnail Image

北京飞米科技有限公司召回4000件无人机4K版套装

2021-07-16
东方财富网
Why's our monitor labelling this an incident or hazard?
The drones are AI systems as they include flight control software that autonomously manages flight parameters. The malfunction (software control parameter mismatch) directly creates a safety risk of drones falling and causing injury, which is a harm to persons. The recall and software update are responses to this malfunction. Since the harm is a plausible direct consequence of the AI system's malfunction, this qualifies as an AI Incident.
Thumbnail Image

4000台小米无人机被召回:会突然坠落砸人

2021-07-16
驱动之家
Why's our monitor labelling this an incident or hazard?
The drones are AI systems as they include flight control software that autonomously manages flight parameters. The malfunction (software control parameter mismatch) could directly lead to physical harm (injury from falling drones). Since the harm is a plausible and direct safety risk caused by the AI system's malfunction, this qualifies as an AI Incident. The recall and software update are responses to an already identified safety hazard with potential realized harm if not addressed.
Thumbnail Image

4000台小米无人机4K版被召回:存飞行时失去动力而坠落的隐患

2021-07-16
凤凰网(凤凰新媒体)
Why's our monitor labelling this an incident or hazard?
The drones' flight control system software is an AI system that controls autonomous flight parameters. The software's improper parameter settings have caused a malfunction leading to loss of power and potential drone crashes, which could cause injury. This constitutes a direct AI Incident because the AI system's malfunction has led to a safety hazard with potential physical harm. The recall is a response to this realized harm risk.
Thumbnail Image

存安全隐患 飞米科技召回4000件无人机套装产品

2021-07-17
杭州网
Why's our monitor labelling this an incident or hazard?
The event involves an AI system component, specifically the flight control system software of drones, which is an AI system as it controls autonomous flight behavior. The malfunction (software control parameter mismatch) could directly lead to physical harm (injury from falling drones). Since the harm is a plausible risk and the company is actively recalling and fixing the issue before harm occurs, this qualifies as an AI Hazard rather than an AI Incident because no actual harm has been reported yet, only a safety risk that could plausibly lead to harm.
Thumbnail Image

飞米科技召回4000件Mi牌无人机 为4K版套装产品

2021-07-16
金融界网
Why's our monitor labelling this an incident or hazard?
The recalled product is an AI system (the drone's flight control system software involves AI for autonomous flight control). The malfunction (software control parameter mismatch) could directly lead to physical harm (injury from falling drones). Since the harm is a plausible safety risk identified and the company is taking corrective action before widespread harm occurs, this constitutes an AI Hazard rather than an AI Incident, as no actual harm has been reported yet but there is a credible risk of injury if the issue is not fixed.
Thumbnail Image

4000 台小米无人机 4K 版被召回:存飞行时失去动力而坠落的隐患

2021-07-16
163.com
Why's our monitor labelling this an incident or hazard?
The flight control system software in the drones is an AI system as it autonomously manages flight parameters. The reported software parameter mismatch is a malfunction of this AI system that could cause the drones to lose power and crash, posing direct physical harm risks. The recall and OTA software update are responses to this AI Incident. Since the harm is realized as a safety hazard with potential injury, this is not merely a hazard or complementary information but an AI Incident.
Thumbnail Image

存在隐患,部分小米无人机4K版被召回

2021-07-19
环球网
Why's our monitor labelling this an incident or hazard?
The event involves an AI system component, specifically the flight control system software of drones, which can be reasonably inferred to include AI for autonomous or semi-autonomous flight control. The malfunction (software control parameter mismatch) could directly lead to physical harm (injury from drone crashes). Since the harm is plausible and the recall is a mitigation effort before widespread harm occurs, this qualifies as an AI Hazard rather than an AI Incident because no actual harm has been reported yet, only a potential safety risk.